Jump the Queue: Effortless Async Task Processing
Architectural elegance for the beginning of a good adventure
PHP’s fpm module has an interesting function: fastcgi_finish_request (https://www.php.net/manual/en/function.fastcgi-finish-request.php).
This function flushes all response data to the client and finishes the request. This allows for time consuming tasks to be performed without leaving the connection to the client open.
This is really interesting, because such a mechanism obviates the need for a full-blown queue for asynchronous task processing on behalf of any given user request. It will only scale to the max number of children (approximately threads, each with it’s own memory limit, see https://stackoverflow.com/a/68017320). In instances where you don't have excessively high traffic volume, this is a pretty handy function to have around. Naturally with any language that has real os-level threads this will be straightforward (e.g. Java, Go, Rust). Even still.. I do love Python, but as far I know, Python doesn't have anything equivalent in terms of simplicity + function (this would change if the elimination of the GIL comes to pass - https://developer.vonage.com/en/blog/removing-pythons-gil-its-happening).
There's something to be said for practical, easy-to-use solutions, especially for asynchronous tssk processing. Not needing to operate a queueing system for a simple web service can be very appealing:
It's less work to develop up front
It can fail relatively gracefully.
For example, when traffic volume is excessive, the thing will not work well, and then should naturally recover when traffic dies down without any administrative intervention. Even if the host machine runs out of disk space, the service shouldn't lan into an unrecoverable state where it needs to be restarted.
People love to hate on PHP, but in some spots it shines. It's been used to build the cores of trillions of dollars worth of companies.
<!— Full disclosure: I don't program much in PHP anymore, but still appreciate it —>