Postgresql service not running

Nextcloud is unavailable, I narrowed it down to the postgresql service not running but I don’t know what the root cause is. I didn’t upgrade anything and didn’t change any configuration, and the failure happened after a week of uninterrupted service

root@syncloud:~# systemctl status snap.nextcloud.postgresql.service
● snap.nextcloud.postgresql.service - Service for snap application nextcloud.postgresql
   Loaded: loaded (/etc/systemd/system/snap.nextcloud.postgresql.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2025-01-01 13:39:11 UTC; 1min 6s ago
  Process: 26632 ExecStart=/usr/bin/snap run nextcloud.postgresql (code=exited, status=1/FAILURE)

Jan 01 13:39:11 syncloud systemd[1]: snap.nextcloud.postgresql.service: Service RestartSec=100ms expired, scheduling restart.
Jan 01 13:39:11 syncloud systemd[1]: snap.nextcloud.postgresql.service: Scheduled restart job, restart counter is at 5.
Jan 01 13:39:11 syncloud systemd[1]: Stopped Service for snap application nextcloud.postgresql.
Jan 01 13:39:11 syncloud systemd[1]: snap.nextcloud.postgresql.service: Start request repeated too quickly.
Jan 01 13:39:11 syncloud systemd[1]: snap.nextcloud.postgresql.service: Failed with result 'exit-code'.
Jan 01 13:39:11 syncloud systemd[1]: Failed to start Service for snap application nextcloud.postgresql.
root@syncloud:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G  5.2G   23G  19% /
devtmpfs        1.7G     0  1.7G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G   33M  1.9G   2% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/loop0      490M  490M     0 100% /snap/nextcloud/812
/dev/loop1      491M  491M     0 100% /snap/nextcloud/809
/dev/loop3      129M  129M     0 100% /snap/platform/2177
/dev/sda        954G  349G  606G  37% /opt/disk/external
/dev/loop4      131M  131M     0 100% /snap/platform/2196
root@syncloud:~# snap run nextcloud.occ
An unhandled exception has been thrown:
Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[08006] [7] could not connect to server: Connection refused
	Is the server running locally and accepting
	connections on Unix domain socket "/var/snap/nextcloud/current/database/.s.PGSQL.5436"? in /snap/nextcloud/812/nextcloud/lib/private/DB/Connection.php:163

In firefox:

Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

I noticed I didn’t have SSH access before I hard reset the Rasberry Pi, now I can at least log in.

Any help is appreciated.

Can you send logs from Settings - Support (tick “include support” first) please?

@boris Done, I sent the logs

Can you run this and post the output here?

systemctl start snap.nextcloud.postgresql

@boris Here you go. I don’t see any useful error message. The exit code is 1 but that doesn’t say much

root@syncloud:~# systemctl start snap.nextcloud.postgresql
Job for snap.nextcloud.postgresql.service failed because the control process exited with error code.
See "systemctl status snap.nextcloud.postgresql.service" and "journalctl -xe" for details.
root@syncloud:~# systemctl status snap.nextcloud.postgresql.service
● snap.nextcloud.postgresql.service - Service for snap application nextcloud.postgresql
   Loaded: loaded (/etc/systemd/system/snap.nextcloud.postgresql.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2025-01-02 10:36:55 UTC; 15s ago
  Process: 5490 ExecStart=/usr/bin/snap run nextcloud.postgresql (code=exited, status=1/FAILURE)

Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Service RestartSec=100ms expired, scheduling restart.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Scheduled restart job, restart counter is at 5.
Jan 02 10:36:55 syncloud systemd[1]: Stopped Service for snap application nextcloud.postgresql.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Start request repeated too quickly.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Failed with result 'exit-code'.
Jan 02 10:36:55 syncloud systemd[1]: Failed to start Service for snap application nextcloud.postgresql
root@syncloud:~# journalctl -xe
-- An ExecStart= process belonging to unit snap.nextcloud.postgresql.service has exited.
-- 
-- The process' exit code is 'exited' and its exit status is 1.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit snap.nextcloud.postgresql.service has entered the 'failed' state with result 'exit-code'.
Jan 02 10:36:55 syncloud systemd[1]: Failed to start Service for snap application nextcloud.postgresql.
-- Subject: A start job for unit snap.nextcloud.postgresql.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit snap.nextcloud.postgresql.service has finished with a failure.
-- 
-- The job identifier is 3045 and the job result is failed.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Service RestartSec=100ms expired, scheduling restart.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Scheduled restart job, restart counter is at 5.
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- Automatic restarting of the unit snap.nextcloud.postgresql.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Jan 02 10:36:55 syncloud systemd[1]: Stopped Service for snap application nextcloud.postgresql.
-- Subject: A stop job for unit snap.nextcloud.postgresql.service has finished
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A stop job for unit snap.nextcloud.postgresql.service has finished.
-- 
-- The job identifier is 3101 and the job result is done.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Start request repeated too quickly.
Jan 02 10:36:55 syncloud systemd[1]: snap.nextcloud.postgresql.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit snap.nextcloud.postgresql.service has entered the 'failed' state with result 'exit-code'.
Jan 02 10:36:55 syncloud systemd[1]: Failed to start Service for snap application nextcloud.postgresql.
-- Subject: A start job for unit snap.nextcloud.postgresql.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit snap.nextcloud.postgresql.service has finished with a failure.
-- 
-- The job identifier is 3101 and the job result is failed.
Jan 02 10:37:07 syncloud Nextcloud[29045]: {"reqId":"a9O4dRVG5TRNTU30UsCt","level":3,"time":"2025-01-02T10:37:07+00:00","remoteAddr":"84.57.212.38","user":"--","app":"index","method":"GET","url":"/index.php/204","message":"{\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\",\"Message\":\"Failed to connect to the database: An exception occurred 
Jan 02 10:37:07 syncloud Nextcloud[29045]: {"reqId":"a9O4dRVG5TRNTU30UsCt","level":3,"time":"2025-01-02T10:37:07+00:00","remoteAddr":"84.57.212.38","user":"--","app":"core","method":"GET","url":"/index.php/204","message":"{\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\",\"Message\":\"Failed to connect to the database: An exception occurred i
Jan 02 10:37:07 syncloud Nextcloud[29045]: {"reqId":"a9O4dRVG5TRNTU30UsCt","level":3,"time":"2025-01-02T10:37:07+00:00","remoteAddr":"84.57.212.38","user":"--","app":"core","method":"GET","url":"/index.php/204","message":"{\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\",\"Message\":\"Failed to connect to the database: An exception occurred i
Jan 02 10:37:07 syncloud ld-2.31.so[960]: syncloud nginx: 84.57.212.38 - niko [02/Jan/2025:10:37:07 +0000] "GET /index.php/204 HTTP/1.1" 500 301 "-" "Mozilla/5.0 (Windows) mirall/3.15.2 (build 20241216) (Nextcloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
Jan 02 10:37:07 syncloud nginx[6519]: syncloud nginx: 84.57.212.38 - niko [02/Jan/2025:10:37:07 +0000] "GET /index.php/204 HTTP/1.1" 500 301 "-" "Mozilla/5.0 (Windows) mirall/3.15.2 (build 20241216) (Nextcloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
Jan 02 10:37:10 syncloud Nextcloud[3032]: {"reqId":"bSLjhwkg0mLHXY4eYHAR","level":3,"time":"2025-01-02T10:37:10+00:00","remoteAddr":"2a02:2488:1c61:9300:5eea:8d3c:f187:cccd","user":"--","app":"remote","method":"GET","url":"/status.php","message":"{\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\",\"Message\":\"Failed to connect to the database
Jan 02 10:37:10 syncloud ld-2.31.so[960]: syncloud nginx: 2a02:2488:1c61:9300:5eea:8d3c:f187:cccd - fred [02/Jan/2025:10:37:10 +0000] "GET /status.php HTTP/1.1" 500 5 "-" "Mozilla/5.0 (Linux) mirall/2.6.2-1build1 (Nextcloud)"
Jan 02 10:37:10 syncloud nginx[6519]: syncloud nginx: 2a02:2488:1c61:9300:5eea:8d3c:f187:cccd - fred [02/Jan/2025:10:37:10 +0000] "GET /status.php HTTP/2.0" 500 0 "-" "Mozilla/5.0 (Linux) mirall/2.6.2-1build1 (Nextcloud)"
Jan 02 10:37:24 syncloud sh[280]: XMT: Forming Solicit, 655350 ms elapsed.
Jan 02 10:37:24 syncloud sh[280]: XMT:  X-- IA_PD dd:97:2b:90
Jan 02 10:37:24 syncloud sh[280]: XMT:  | X-- Request renew in  +3600
Jan 02 10:37:24 syncloud sh[280]: XMT:  | X-- Request rebind in +5400
Jan 02 10:37:24 syncloud sh[280]: XMT: Solicit on eth0, interval 126470ms.
Jan 02 10:37:24 syncloud dhclient[705]: XMT: Solicit on eth0, interval 126470ms.
Jan 02 10:37:24 syncloud dhclient[705]: RCV: Advertise message on eth0 from fe80::1eed:6fff:fe9a:d91f.
Jan 02 10:37:24 syncloud sh[280]: RCV: Advertise message on eth0 from fe80::1eed:6fff:fe9a:d91f.
Jan 02 10:37:24 syncloud sh[280]: RCV:  X-- Server ID: 00:03:00:01:1c:ed:6f:9a:d9:1f
Jan 02 10:37:24 syncloud sh[280]: PRC: Lease failed to satisfy.

Can you try this

snap run nextcloud.postgresql

and then this

SNAP_DATA= /var/snap/nextcloud/current /snap/nextcloud/current/bin/service.postgresql.sh start

@boris Both fail with the same error. I only have one account root. Should I create another user?

root@syncloud:~# snap run nextcloud.postgresql
pg_ctl: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
root@syncloud:~# users
root
root@syncloud:~# SNAP_DATA=/var/snap/nextcloud/current /snap/nextcloud/current/bin/service.postgresql.sh start
pg_ctl: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.

Sorry try this:

sudo -u nextcloud SNAP_DATA= /var/snap/nextcloud/current /snap/nextcloud/current/bin/service.postgresql.sh start

@boris Now we get some useful output

root@syncloud:/tmp# sudo -u nextcloud SNAP_DATA=/var/snap/nextcloud/current /snap/nextcloud/current/bin/service.postgresql.sh start
pg_ctl: another server might be running; trying to start server anyway
2025-01-04 13:52:05.748 GMT [11336] FATAL:  lock file "postmaster.pid" already exists
2025-01-04 13:52:05.748 GMT [11336] HINT:  Is another postmaster (PID 534) running in data directory "/var/snap/nextcloud/current/database"?
pg_ctl: could not start server
Examine the log output.
root@syncloud:/tmp# cat /var/snap/nextcloud/current/database/postmaster.pid 
534
/var/snap/nextcloud/current/database
1550139132
5436
/var/snap/nextcloud/current/database

   137695         0
ready  

And indeed there is a process 534:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                   
  534 nextclo+  20   0   61064   9336   6796 S   0.3   0.2   2:44.54 jemalloc_bg_thd

I stopped and started it again

root@syncloud:/tmp# sudo -u nextcloud SNAP_DATA=/var/snap/nextcloud/current /snap/nextcloud/current/bin/service.postgresql.sh stop
pg_ctl: server does not shut down

But now 534 is gone. And I can start

root@syncloud:/tmp# sudo -u nextcloud SNAP_DATA=/var/snap/nextcloud/current /snap/nextcloud/current/bin/service.postgresql.sh start
pg_ctl: another server might be running; trying to start server anyway
2025-01-04 13:54:39.598 GMT [11420] LOG:  ending log output to stderr
2025-01-04 13:54:39.598 GMT [11420] HINT:  Future log output will go to log destination "syslog".

Now Nextcloud is accessible again. Great.

I don’t want to do these manual steps each time I restart the server, how can I achieve that?

I think the problem happens during the upgrade, can you run:

snap refresh nextcloud

@boris Is the nextcloud snap updated automatically? Sorry if this is a noob question. I don’t recall updating it myself. Here is the output, nothing to update at the moment

oot@syncloud:~# snap refresh nextcloud
snap "nextcloud" has no updates available
root@syncloud:~# snap list nextcloud
Name       Version  Rev  Tracking       Publisher  Notes
nextcloud  820      820  latest/stable  syncloud✓  -

Yes apps are automatically updated, it is much easier to support users that way, especially when nextcloud cannot skip major version upgrades.

Can you reboot to see if postgres starts automatically?