mercredi 4 février 2015

Send heavy computation jobs from web server to another process and wait for response


I know it is a common practice to send cpu-bounded heavy tasks to background processes, in order to free a web server to serve other requests.


But - what if the request must know the result of that computation? Is there a known pattern, where the web server waits for a background process to finish its job, before it responds? I couldn't find any example of this use case.


In my case, for instance, I want to send an HTTP post request's body to Spark Streaming process that runs separately from the server, wait for the result and then respond to the request. (Server and Spark Streaming communicate through Kafka)


Many thanks





Aucun commentaire:

Enregistrer un commentaire