Data flows into my Scala program in the form of a stream of events (case classes), and I'd like to commit this data into my database (Neo4J) in batches, to avoid starting a transaction for each element of data.
I'd like a batch to be written whenever it exceeds a size, or an age in milliseconds
val batcher = SomeKindOfBatcher[MyEvent](maxSize = 100, maxAge = 1 second)
batcher.onBatch(db.writeBatch) // writeBatch accepts a Seq[MyEvent]
while(input.hasNext()) {
batcher.enqueue(input.next())
}
I'm looking for the most appropriate library that provides an interface like SomeKindOfBatcher above
Aucun commentaire:
Enregistrer un commentaire