3

I am new to NoSQL DBs, so please forgive my ignorance.

I am looking for a distributed object store that would make it easy to implement hierarchical purging. What I mean by that is that conceptually objects/documents would be put in a folder-like structure that is either part of or separate from their key. When a distributed process completes a certain task or unit of work, it may want to purge all of the data associated with this unit of work -- which may be in a "folder".

One idea is to have a key that is based on a URI. For example, an object my have a key something like:

/System/systemId/Process/processId/Task/taskId/MyParentObj/parentId/MyObj/objId

Then, when I want to purge all of the data for, say my task, I can search for all of the objects whose keys start with /MySystem/systemId/MyProcess/processId/MyTask/taskId and delete them.

Is there a NoSQL DB out there (with a .NET client API) that natively supports this kind of structure without having to scan all the objects in the system?

We are potentially storing terabytes of temporary data during a single distributed process, so memory-only databases wouldn't work for us.

We would also like to do some querying and enumerations of our objects based on hierarchy.

1
1

Without knowing more about your use cases, it is pretty hard to make a proper suggestion here. Quite some DBMS fit the use case you describe, but may be totally wrong for other use cases.

Furthermore, I have to add that BigData requires some in-depth knowledge and experience to set up the chosen DBMS properly. Please make sure that you understand the ramifications when selecting one DBMS.

Since you do not state what else you want to do with the database, I can only give you a rough, very high level overview.

In general, you have two main options, imho. Either you can use a graph based database or a document oriented database. While it would surely be possible to model hierarchical data in a lot of other database types, it will be incomparably more difficult to do so efficiently.

There is a good explanation of various methods to model tree structures with MongoDB. I am not too sure wether this can be applied to CouchDB or other document based DBMS easily, though.

I would like to note that you can have the best of both worlds combined, if you are a bit adventurous: Cayley, a graph database, supports multiple backends, among them MongoDB. You could access the Graph data via Cayley's REST API or a client, with the best supported one being the Go client (Cayley is written in Go). Plus, you'd have a document oriented database deployed.

Please note that Cayley is quite a bit hacky, as of now. But for simple applications of graph access such as yours, it should do the job well.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.