I would like to use the multiprocessing library to speed up the time it takes to insert data values into my mongo database.
I found from the following resource: http://ift.tt/1LmaykF
Some nifty results performance wise - but I grew concerned about data integrity and the possibility of bulk inserts stepping on each other if they are comming from different processes around the same time? Is that a valid concern with respect to pyMongo?
I am essentially reaching out to a device, parsing data from them, and writing to a single database using a bulk insert. I scale each host I reach out over multiple processes (via the multiprocessing library) at the same time.
I don't have a lot of experiance with this. So I am seeking guideance to see if I need to be concerned.
Thanks!
Aucun commentaire:
Enregistrer un commentaire