Recently we encountered some strange issues with httpd on some of our CentOS 7 machines during my current project.

Through our pipeline we restart httpd several times which sometimes leads to this error:

Apache: [error] (28)No space left on device

After some research we found out the semaphores were all being used blocking httpd daemon to restart.

The list of semaphores can be fetched by issuing

# ipcs -st

------ Semaphore Operation/Change Times --------
semid    owner      last-op                    last-changed
753664   apache      Not set                    Mon Feb 17 20:20:47 2020
786433   apache      Not set                    Mon Feb 17 20:20:47 …