tag:blogger.com,1999:blog-6569681.post3001013896037119258..comments2024-01-15T13:17:33.771-08:00Comments on Geeking with Greg: Cassandra data store at FacebookGreg Lindenhttp://www.blogger.com/profile/09216403000599463072noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-6569681.post-59278186334471939802008-08-20T10:52:00.000-07:002008-08-20T10:52:00.000-07:00Hi Greg , This is implemented and is exposed via...Hi Greg ,<BR/><BR/> This is implemented and is exposed via the JAVA API , but not all of the thrift API's , for example if you look at batch_insert_blocking , it implements these semantics.<BR/><BR/>- PrashantPrashant Malikhttps://www.blogger.com/profile/13739897608879163700noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-13976137653937896142008-08-20T07:15:00.000-07:002008-08-20T07:15:00.000-07:00Hi, Prashant! Thanks for the clarification! That...Hi, Prashant! Thanks for the clarification! That all sounds like the right way to go.<BR/><BR/>On the option of waiting for a majority of N replicas to return successfully, the wiki <A HREF="http://code.google.com/p/the-cassandra-project/wiki/WritePath" REL="nofollow">says</A>, "The comments indicate that we wait for a reply from 'X <= N' endpoints, but I don't see this in the code." I took that to mean that the wait for the majority of N nodes was not implemented. Is that incorrect?<BR/><BR/>Thanks again, Prashant!Greg Lindenhttps://www.blogger.com/profile/09216403000599463072noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-15284797860365890072008-08-19T23:50:00.000-07:002008-08-19T23:50:00.000-07:00The writes to the replicas are done immediately in...The writes to the replicas are done immediately in an asynchronous manner.<BR/>The client has the option of using an API to do blocking inserts which wait for the data to be written to a quorum or a majority of the replicas before returning.<BR/><BR/>In case of machine failures the data is repaired using techniques like read repair and hinting once the machine comes back up.Prashant Malikhttps://www.blogger.com/profile/13739897608879163700noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-37869251596175687132008-08-18T18:24:00.000-07:002008-08-18T18:24:00.000-07:00Hi, Avinash. Thanks for coming by!Ah, yes, I see i...Hi, Avinash. Thanks for coming by!<BR/><BR/>Ah, yes, I see in the slides a note that writes do go out to a transaction log immediately before going to memory. Then, later, the disk-based tables are updated in batch, right.<BR/><BR/>Sorry about the confusion, I'll update the post to clarify.<BR/><BR/>While you're here, can you answer another question? When are updates to replicas done? That doesn't seem to be clear from the slides or wiki. I'm curious how much of a window there might be for data loss if a machine croaks and doesn't come back up?Greg Lindenhttps://www.blogger.com/profile/09216403000599463072noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-54207522506617312162008-08-18T15:53:00.000-07:002008-08-18T15:53:00.000-07:00Writes are not cavalier. Every write is logged int...Writes are not cavalier. Every write is logged into a Commit Log at each replica. Only if this write is successful does the local replica update the in-memory copy. This makes sure that in case a server/replica crashes then it can be recovered from the Commit Log.Phantomhttps://www.blogger.com/profile/05105215151973404168noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-73601378678448213832008-08-16T01:16:00.000-07:002008-08-16T01:16:00.000-07:00How does this compare with Hadoop?How does this compare with Hadoop?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6569681.post-24796027752742372992008-08-15T20:44:00.000-07:002008-08-15T20:44:00.000-07:00Hey Greg - Not sure if you take "requests", but I'...Hey Greg - Not sure if you take "requests", but I'd love to hear your opinions and insights on triple stores (in the semantic web sense). You have any experience? Systems like Cassandra, BigTable, or Dynamo seem like the early throes of triple stores - they're only missing one more attribute.Jontehttps://www.blogger.com/profile/01224210196129491075noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-66222506053775187782008-08-15T18:25:00.000-07:002008-08-15T18:25:00.000-07:00Hi, Jeff. Thanks for coming by to chat about this...Hi, Jeff. Thanks for coming by to chat about this!<BR/><BR/>The part I'm confused about is what happens if you make a write, then the machine goes down before the commit log is written to disk. Is there a window where data loss could occur?<BR/><BR/>On waiting for a majority of N nodes to return successfully, the wiki (which it appears you wrote) <A HREF="http://code.google.com/p/the-cassandra-project/wiki/WritePath" REL="nofollow">says</A>, "The comments indicate that we wait for a reply from 'X <= N' endpoints, but I don't see this in the code." I took that to mean that the wait for the majority of N nodes was not implemented. Is that incorrect?<BR/><BR/>Thanks, Jeff!Greg Lindenhttps://www.blogger.com/profile/09216403000599463072noreply@blogger.comtag:blogger.com,1999:blog-6569681.post-76157297030846976562008-08-15T18:06:00.000-07:002008-08-15T18:06:00.000-07:00hey greg,actually, cassandra has a configurable co...hey greg,<BR/><BR/>actually, cassandra has a configurable consistency model for writes. if throughput is a priority, you can dispatch writes asynchronously to the n nodes managing the n replicas. failures will be caught on the next read. you can also execute a blocking write and wait for a majority of the n nodes to return successfully. either way, all writes persist to a redo log in addition to being inserted into the in-memory table structure.<BR/><BR/>also, if any of the nodes that manage the key or its replicas are down at the time of a write, the system will find another node to take the write, and take note of the downed node. if that node comes up, the data will be passed back to its appropriate owner. this is the "hinted handoff" algorithm detailed in the dynamo paper.<BR/><BR/>via the redo log and hinted handoff, the system is designed to never lose a write.<BR/><BR/>in addition, we're working to add support for more stringent consistency models, similar to the work done with megastore.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6569681.post-84057631844825164272008-08-15T17:53:00.000-07:002008-08-15T17:53:00.000-07:00Hi!Have you looked at http://hypertable.org/ ?Are ...Hi!<BR/><BR/>Have you looked at http://hypertable.org/ ?<BR/><BR/>Are you still working from home? Ping me if you want to go get some tea.<BR/><BR/>Cheers,<BR/> -BrianAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-6569681.post-5231312648306360992008-08-15T17:48:00.000-07:002008-08-15T17:48:00.000-07:00My first impression of Cassandra is that it is the...My first impression of Cassandra is that it is the love-child of Bigtable and Dynamo. You have Dynamo's gossip protocols and consistent hashing instead of BigTable's table servers. However, unlike Dynamo, where you can tune the number of writers who need to ack a write , Cassandra does seem a bit cavalier about writes.Sriram Krishnanhttps://www.blogger.com/profile/06436908688620816300noreply@blogger.com