We began testing our work in this area by starting with our load balancing policies. For the Java Driver, we use the LoadBalancingPolicy interface and provide three load balancing policies by default: RoundRobinPolicy (default), TokenAwarePolicy, and DCAwareRoundRobinPolicy.

RoundRobinPolicy

For the RoundRobinPolicy, testing was decently straightforward. We not only tested that each subsequent request was spread out amongst all active nodes, but also that the active node list was accurate and actively updated after topological changes. This can be done without worry of where the final location of the data will be, since Cassandra implements coordinator roles when issuing requests.

The client can connect to any cluster member, which acts as a coordinator for the requests it receives. In the event that the request fails, or the acting coordinator node dies, a subsequent request using the RoundRobinPolicy will choose another peer node as a coordinator and should complete the request successfully once the specified ConsistencyLevels are met.

TokenAwarePolicy

The TokenAwarePolicy is a more interesting case since this policy also requires a child policy that dictates which nodes are local nodes and which are remote nodes, as far as the driver is concerned. For cases with a single datacenter, the TokenAwarePolicy chooses the primary replica to be the chosen coordinator in hopes of cutting down latency by avoiding the typical coordinator-replica hop.

To test this policy we tried multiple setups of different child policies as well as multiple datacenters, some of which were programmed to disappear altogether, simulating full datacenter outages. Although we did find possible minor issues in this area, the fail-over seemed to work great and as expected. If anything, fail-over optimizations may still be made, but two things are for certain: token aware optimization works and fail-over scenarios continued without a hiccup.

Cassandra’s token policy is handled innately, abstracted away from the user, in order to provide a simple and robust environment for both operations and developer teams. It’s this same simple and robust consistent hashing that allows for the TokenAwarePolicy to optimize requests by contacting the ideal node. And of course, if this ever fails, the specified child policy becomes the active LoadBalancingPolicy as the search for a replica that is both alive and responsive continues.

DCAwareRoundRobinPolicy

The DCAwareRoundRobinPolicy takes the same fundamentals as the RoundRobinPolicy, but introduces support for multiple datacenters. To test this policy we followed suit with implementing full datacenter outage cases and fail-over assertions when testing this policy, much like our TokenAwarePolicy tests. Although this may not initially seem highly beneficial, the case when datacenters exist in different parts of the world highlights why this is the best policy when dealing with remote datacenters.

Let’s take the use case of a Cassandra cluster that is split up into 3 Amazon regions: US-East, US-West, and EU-West with a keyspace replication_strategy of {‘US-East’: 3, ‘US-West’: 1, ‘EU-West’: 1}, where US-West and EU-West are designed to primarily be for backup or fail-over scenarios for the given keyspace.

In the ideal situation, you wouldn’t want to spread your requests across a WAN since a climb in latency will be apparent for requests that have to cross the Atlantic Ocean. Instead, you would want to constrain your application to contact only the nodes on the LAN to cut down on this latency.

This does not mean that data will only stay in US-East, for this specific replication_strategy, but as far as your driver is concerned, it’s the primary datacenter. In the background, data is being migrated in batch for every write without developer interaction. (Although from an operations standpoint, routine repairs are required to guarantee consistency.)

Even if datacenters were not physically separated by great distances, it’s best to choose a single datacenter and continue to write to just that one. This ensures all requests made to the cluster, for a given session, will work over the same data. This is the way that fail-over works on the DCAwareRoundRobinPolicy as well. If US-East ever falls over, you’re guaranteed to choose a single datacenter as the interim primary datacenter until your primary datacenter returns, thus providing a higher level of consistency at a lower latency. The DCAwareRoundRobinPolicy takes all this into consideration so that even new users can easily store data efficiently, without worrying too much on “how” it’s being stored since this has already been implemented and provided as part of the driver.

What’s Next?

For our next major release, we’ve already begun our Jenkins integration which should help us ensure that our test suite, which already has a runtime of more than an hour, is run daily against our master branch. This will ensure continuous stability as we grow both our test code and our product code.

We will also be building and ensuring that a long-running duration test shows that the driver continues to be accurate, performant, and stable. Even though a collection of 81 tests, covering 76% of the code, takes a full hour to be validated, these tests are all running on small disposable clusters that are running very specific tests. When we start testing our driver against 72+ hour duration tests, in conjunction with simulated chaotic environments, we will continue to ensure that data is always valid on both sides of the pipe and that our fault-tolerant database has a fault-tolerant client driver that is run on the new binary protocol.

Do continue to look out for updates around our client drivers as we continue to grow our documentation, examples, and our great team.

And if all that’s not enough, you can always meet with us face-to-face at the Cassandra Summit 2013′s “Birds of a Feather” and “Stump the Experts” sessions!

The good news is that we’ve got both bases covered in the executive track of our upcoming Cassandra Summit. This is new for us this year and is happening for a very important reason: you asked for it! At last year’s Summit, we fielded numerous requests for a track dedicated to the business side of NoSQL technology, and I guarantee that what we have in store for you at this year’s event won’t disappoint.

The fact that Cassandra has gone mainstream can be seen in the speaker lineup that makes up this year’s Summit. In the executive track, we have a great set of tech execs who will be telling you both why they chose NoSQL and how they successfully implemented it in their organization.

Although all the talks look great, I thought I’d call out three of the ones I won’t be missing:

1. Taking Risks Without Risking Your Career, Christos Kalantzis, Netflix. Netflix was just christened as the biggest cloud application in the world. Netflix also stores 95% of their data on Cassandra today, making the switch from Oracle a few years ago. How does a smart IT exec safely go from tried-and-true, but old-and-won’t-do to new technology that can both transform how a company does business and save a significant sum of money in the process? Few have the kind of success and tell the story better than Christos and Netflix, so who better to learn from?
2. Stop Crippling Your Business: Fundamental Considerations Everyone Needs to Know, Vincent Dell’Anno, Accenture; John Whittaker, Dell. Few execs know how to evade the perils of not creating proper NoSQL standards and how to avoid the unnecessary red tape that can bog down IT projects better than the guys from Accenture and Dell.
3. It’s like your parents: Relational and NoSQL can co-exist, Sean Knapp, Ooyala. Having a co-existence strategy for new and legacy technology is key when it comes to implementing something like NoSQL: use the right tech in the right place at the right time. When you distribute video content for the likes of ESPN and Rolling Stone, plus take in and analyze literally ¼ of all the video views on the Internet like Ooyala does, you learn a thing or two about how to smartly put the right puzzle pieces together so relational and non-relational play nice.

Of course, there are other great exec talks from Splunk, Cowen, Jaspersoft, and more that will help you answer both the “Why?” and “How do I?” questions that IT leaders are asking about NoSQL. Check out the full schedule and make plans to be at this year’s Summit, which will be here before you know it. Register now and save 25% with the promo code SFSummit25.

See you there!

The DataStax ODBC driver for Cassandra was developed by ODBC software leader Simba Corporation and is compliant with the latest ODBC 3.52 specification and runs on both 32 and 64-bit platforms.

Let’s take a quick walk through of how it works.

Installing the DataStax ODBC Driver for Cassandra

The DataStax ODBC driver for Cassandra is currently provided for Windows platforms, and can be downloaded from the DataStax website.

Installing the DataStax ODBC driver for Cassandra is simple. The only prerequisite needed for Windows is to have the Microsoft Visual C++ 2010 runtime installed for the appropriate platform (either 32 or 64-bit).

The installation of the driver is completed in just a few steps.

Note that you may need to install both the 32 and 64-bit versions of the driver if, for example, you have certain tools that are 32-bit but are running on a 64-bit box.

Configuring the DataStax ODBC Driver for Cassandra

Configuring the DataStax ODBC driver for Cassandra is easy. On Windows, you invoke the ODBC Data Source Administrator utility and first validate that the driver is present:

Then, you create either a User or System DSN (data source name) that will be used by front end business intelligence tools. You’ll need to enter a name for the datasource as well as the IP or host name of the Cassandra node you want to connect to, along with a keyspace that will be referenced by the driver. Once the information is entered, you can test your connection to ensure it connect to your cluster.

Using the DataStax ODBC Driver for Cassandra

Note that the current driver does not currently support version 3.0 of the Cassandra Query Language (CQL3), which is the default for Cassandra 1.2 and higher (an upcoming version of the driver will support CQL3). However, CQL2 is supported, so you will want to make sure that any objects you want to reference with the driver are created with CQL2.

The Windows installer for Cassandra 1.2 and higher uses CQL3 by default and creates a shortcut for the CQL utility that can be easily clicked on and run. To create a CQL utility shortcut that uses CQL2, just create a new shortcut with the following target executable (substituting your install location if you changed it from the default location):

“C:\Program Files (x86)\DataStax Community\python\python.exe” “C:\Program Files (x86)\DataStax Community\apache-cassandra\bin\cqlsh” -2

Once you’ve successfully configured your ODBC datasource, you can use it in your preferred BI tools to connect to and pull data back from Cassandra. For example, to use the DataStax ODBC driver for Cassandra with Microsoft Excel, you can use the data connection wizard to select your new ODBC datasource:

And then, select one or more data objects to pull back data:

The ODBC driver also works with generic ODBC dev/query tools like Query Tool ODBC:

Conclusions

Currently the DataStax ODBC Driver for Cassandra is in beta, but will be GA shortly. For more information, visit the client drivers downloads page on the DataStax website.

Thrift, the transport layer that Cassandra traditionally used for client-server communication, has been a great opportunity for the database in its early days since Thrift clients were readily available for most programming languages. This allowed the Apache Cassandra project to focus on the server side without spreading their resources on the client side. Nevertheless, as Cassandra became mature, Thrift turned out to be a limitation: communication limited to the request-response paradigm, no notifications, no streaming, client side interface made of generated code, etc. In Cassandra 1.2, a solution to this second problem was introduced with the CQL Native Protocol, a protocol designed exclusively for CQL3 and with enough flexibility to enable new features in Cassandra for the years to come.

With these two majors changes, an update was obviously necessary on the client side. The fact that we had a renewed interface and transport layer, together with the need for an environment free of any Thrift concepts, strongly advocated for a brand new line of drivers, which was aligned with the long-term strategy of CQL.

Today, DataStax announces version 1.0.0 of a new Java Driver, designed for CQL and based on years of experience within the Cassandra community. This Java driver is a first step; an object mapping and a JDBC extension will be available soon, drivers for C# and other major languages are on their way. Besides this new interface and API, this new driver comes with:

• Node discovery, load balancing, and fail-over are implemented in a standardized way across all languages
• Asynchronous API, making it simple to send multiple requests in parallel
• Query builder, which makes it possible to create queries programmatically
• Tracing, a new feature in Cassandra 1.2, that will quickly become part of everyday tools for developers working with CQL

In the weeks to come, several articles on this blog will cover some particular features of this Java Driver. Meanwhile, its documentation is readily available and you can start to use it right away using the following Maven dependency:

<dependency>
   <groupId>com.datastax.cassandra</groupId>
   <artifactId>cassandra-driver-core</artifactId>
   <version>1.0.0</version>
</dependency>

Our ODBC driver for Hive has been available for a while now, however while we charged a small fee for it in the past, we’re now making it completely free for everyone to use.

Something new is our ODBC driver for Cassandra, which we’re providing in beta form right now for the Windows platform. Our new Cassandra driver conforms to the standard ODBC 3.52 standards, which practically means you can use it to connect to any open source Cassandra or DataStax Enterprise cluster from many open source and proprietary BI, query, and ETL tools (e.g. Microsoft Excel, Tableau, MicroStrategy, etc.), and work with data in Cassandra.

Again, note that the Cassandra driver is beta at the moment, so be sure to read the accompanying documentation that comes in the installation package to understand the current limitations (e.g. no support for CQL3 at the moment; a future release will work with CQL3).

You can download the ODBC drivers now on our drivers downloads page, and don’t hesitate to contact us if you have any questions.

Highlights include:

• Full documentation
• Out-of-the-box best practices for node discovery, load balancing and fail over
• An asynchronous architecture provides simpler concurrency without any thread pool tuning
• Tracing support
• CQL with prepared statements is about 10% faster than Thrift, and we expect that gap to widen as the native protocol matures

More qualitatively but perhaps even more important, this addresses the paradox of choice we’ve had in the Cassandra Java world: multiple driver choices provide another barrier to newcomers, where each must evaluate the options for applicability to his project. Having just done such an evaluation to settle on Cassandra itself, this is the last thing they want to spend time on.

And that’s the best-case scenario. More often, a fragmented landscape leads to many solutions, each of which solve a different 80% of the problem. Better to have a single, well-thought-out solution, that lets people get started writing their application immediately. The native CQL driver provides exactly that.

Get the native CQL driver here. Want to learn more? Check out Patrick McFadin’s talks on data modeling and the new drivers at the 2013 Cassandra Summit, June 11-12. Register today with the code SFSummit25 for a 25% discount, and check out the ten talks I’m most looking forward to.

I’ve been to a lot of conferences in my 20+ years in the database industry, and I cannot remember one that had a more impressive lineup of speakers, including: Accenture, Barracuda Networks, Blue Mountain Capital, Comcast, Constant Contact, Dell, eBay, Fusion-io, Intuit, Microsoft, Netflix, Sony, Splunk, Spotify, Walmart, and many more, all at one event to share their Cassandra experience.

And for the first time, we’ve created an “Executive Track” that will deal specifically with business and personal/career implications associated with implementing a new technology.

The audience will be diverse (developers, architects, DBAs, executives), the topics will be meaningful, and the talk will be straight.  We would love to have you there!  You can register here and use the code SFSummit25 for a 25% discount!

OpsCenter Enterprise edition is free to try in development environments, so try it now.

Here are some of the talks that I’m personally most excited to see:

1. Time for a new relationship – Intuit’s journey from RDBMS to Cassandra: I’m leading with this one on purpose, because I can’t wait to see Intuit’s description from the trenches of migrating from an RDBMS to Cassandra. Remember when the conventional wisdom was that you couldn’t trust financial data to anything but a relational database? Step one is not losing data, and durability has been designed in to Cassandra from the start, unlike much of the competition. Step two is living without the rest of ACID, and Cassandra experts like Matt Dennis have been showing how to apply eventual consistency to financial data for some time. Understanding the advantages this generates for business is starting to hit the mainstream. Intuit’s talk will help lay this myth to rest for good.
2. Building a Scalable Time-Series Database with Cassandra: Expect deep technical information on how to get the most performance possible out of a production workload, updated from Jake and Carl’s well-done talk from NYC* earlier this year. This talk at the Summit will include more performance tuning results, a discussion of upgrading Cassandra live, and generalizing their data store to be a generic object store instead of just time series data. Also of note: this is one of the first production applications built entirely on CQL.
3. Buy It Now! Cassandra at eBay: Jay gave one of the best talks at last year’s Summit. I’m excited to see what his team has done since then and what lessons he has to share. Related: Managing Cassandra at eBay Scale.
4. Suicide Prevention Using Social Media and Cassandra. Big data has been dismissively summarized as figuring out how to get people to click on more ads. Definitely a feel-good story for me to see a counterpoint like this.
5. Migrating from MySQL to Cassandra: Michael upgraded his production cluster at Barracuda Networks to Cassandra 1.2.0 the day it was released. I’m pretty sure that’s what Nietzsche had in mind when he said, “That which does not kill me makes me stronger;” I’m looking forward to Michael’s war stories.
6. The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop: I never thought I’d see the words “Hector,” “IKVM,” and “production” in the same sentence. Someone should make this into a T-shirt. It’s almost a shame that the Native CQL .NET driver is making this stack obsolete.
7. Pushing Cassandra’s Boundaries: (Re)-Building the Social Grid for Global Telcos @ 1/10th the Market Cost: Openwave has been pushing Cassandra hard with their messaging suite for a couple years now. They’ve been especially active in pushing the envelope on dense, many-TB-per-machine deployments. This is a space that’s seeing increasing attention this year, so I plan to pay attention to pioneers like Openwave.
8. Real-time Analytics Using Cassandra and Spark/Shark: Ooyala is another experienced Cassandra shop, and has built their own inputformat that understands their custom indexing solution. The Shark driver then uses that. Very interesting, especially if you’re looking for examples of what you can do with open APIs and some custom code. Ooyala is also presenting the talk on Linux and Cassandra Tuning.
9. The State of CQL: A deep dive into modern CQL, by Sylvain Lebresene, the man who has written most of it. This is the future of Cassandra. Also recommended: Patrick McFadin’s talk, The World’s Next Top Data Model. Patrick has been teaching CQL data modeling for longer than anyone and is on a mission to demystify CQL from a user’s point of view; after this session, you will be ready to be dangerous with the new CQL drivers.
10. Last but not least, Distributed Graph Computing with Titan and Faunus: graph computing is still a bit niche, but a growing one. Titan is a distributed graph database built on Cassandra, integrated with the popular TinkerPop graph stack. Titan is making rapid progress and already has production users.

Cassandra is going mainstream, and we expect an even more diverse crowd at this year’s Summit. Besides Cassandra experts, we will see many new developers as well as relational DBAs, managers, and executives. Check out the sessions and schedule pages to see the content we have lined up for these audiences as well. Register today with the code SFSummit25 for a 25% discount!

• Graph-based Recommendation Systems at eBay manages to give both a good overview of recommendation systems and an interesting Cassandra use case. Frankly, my experience is that most “Cassandra in Field X” talks manage to give a good Cassandra talk, or a good introduction to X, but not both. Thomas Pinckney’s talk proved a happy exception. (During the Q&A, you can hear me ask why eBay doesn’t use item-to-item collaborative filtering, which to my novice’s perspective appears to be more popular currently. I’ll leave you to the talk for Thomas’s answer.)
• If you’re at all interested in Storm, Kafka, or Elastic Search, I’d recommend Brian O’Neill’s talk on integrating all three with Cassandra. I think Brian’s just a little ahead of the curve here, and you’re going to see a lot more from Storm and Kafka in particular this year.
• DataStax lost Jake Luciani as an employee to Blue Mountain Capital last year. I was sorry to see him go, but this does leave the world with one more Cassandra committer building applications in the trenches. I love seeing that because these people have the mentality not just of “how can I work around this problem I ran into with Cassandra,” but also “how can I make Cassandra better to solve this problem for everyone?” Jake’s talk with Carl Yesigian is a great example of applying this to time series financial data on the cutting edge of Cassandra 1.2. (And for a good overview of what’s new in 1.2, I’ll refer you to my own talk from NYC*.)
• Nathan Milford’s talk on Cassandra administration was a good change of pace from the talks focused on architects and developers. Even if you aren’t at all interested in sysadmin procedures, though, you need to watch this for Nathan’s unique introduction. I’ve never seen anything like it at a tech conference.
• Last but not least, you should definitely check out Michaël Figuière’s talk on DataStax’s new CQL drivers for Java and C#. CQL is a huge step forward and having drivers that speak it natively are going to make a night-and-day difference in Cassandra development productivity. Check it out.

We already have more great talks lined up for the two-day Cassandra Summit in June. Register with the code SFSummit25 for a 25% discount!

But, there are cases when the model above doesn’t provide enough performance, more specifically:

1. Reindexing: DSE Search data can be reindexed “in place”, without forcing the client to resubmit data; this happens by going through Cassandra SSTables and sequentially indexing all rows.
2. Repairing: Cassandra data can be repaired via the nodetool command, which triggers the reindexing of actually repaired rows by sequentially going through them.
3. Bulk loading: many users implement home-made bulk loading solutions, which usually provide limited concurrency.

Multi-threaded indexing in DSE Search 3.0.1 comes to help in such a cases, also providing a general, improved, concurrent indexing model.

The new indexing model decouples writes to Cassandra from writes to the Lucene index, making the latter asynchronous. Conceptually speaking, it simply works as follows: when a document is inserted, its data is written into Cassandra and the indexing request is queued-up to be asynchronously processed by a pool of worker threads; at given intervals, when Memtable data is flushed from inside Cassandra, or a commit happens from inside Solr, the indexing queue is synchronously flushed, so that all in-flight indexing requests become visible and all committed data up to that point can be queried.

As simple as it sounds, when implementing asynchronous work models a few important problems have to be addressed:

1. Concurrent execution of the logically same work unit: i.e., what if two asynchronous threads try to index the same document?
2. Flow control between work producers and consumers: i.e., what if indexing requests are submitted faster than they are processed?
3. Visibility and management: how can I know about my producers and consumers work? How can I tune it?

Let’s see how DSE Search 3.0.1 solves all of them.

First, we implement per-document thread affinity to avoid concurrent indexing of the same document: we hash the document identifier and assign the document to an in-memory queue serving one and only one indexing thread; by doing so, we partition the work between different threads, and make sure to always process indexing requests for the same document from the same thread.

Flow control is implemented via automatic back-pressure: at Cassandra flush or Solr commit time, we compute some heuristics representing the current load of the indexing system, based on index processing time and indexing queue depth, and if a configurable threshold is met, we pause producers (that is, incoming indexing requests) until all accumulated in-flight indexing requests are processed: please note this is different from a normal “flush situation”, when newly arrived indexing requests are allowed to queue-up and in-flight requests are flushed only up to the current point in time.

Visibility and management are implemented via configuration switches and JMX mbeans. First, you can configure the maximum number of indexing threads per core with the following dse.yaml setting: max_solr_concurrency_per_core; the default value is computed based on the available number of CPUs (which is usually a good value to stick on), but you can configure it to best suit your indexing volume, as well as set to 1 to get back to the old synchronous behavior. Then, for each Solr core, you have several configuration knobs and monitoring gauges available via the IndexPool-core_name mbean (where core_name is an actual Solr core name) in the com.datastax.bdp domain, with the more relevant ones being:

• BackPressureThreshold: the (average) max number of queued documents that will trigger the back-pressure mechanism discussed above; this is in other words a way to control and limit memory consumption of the whole indexing system.
• MaxConcurrency: the maximum number of indexing threads, between the previously mentioned max_solr_concurrency_per_core and 1; this way you can dynamically adjust your concurrency level, and even get back to the old synchronous model, without restarting.
• QueueDepth: current depth of all queues.
• TaskProcessingTime: time it took to process the latest indexing request, per queue/thread.
• ProcessedTasks: total number of processed tasks, per queue/thread.

Finally, some performance considerations: in our tests we’ve got a 30%-40% performance increase in sequential indexing use cases like repair and reindex, but it’s all about your use case in the end. With the visibility and configurability discussed before you have (hopefully) all the right tools for a correct analysis and effective tuning, but if in need of any help, do not hesitate to reach us via our public forums.

But the eWeek article demonstrates that the same concerns exist where Hadoop implementations are concerned. The article says: “It [Hadoop] was not written to support hardened security, compliance, encryption, policy enablement and risk management.”

Because DSE is a single integrated platform that includes Apache Cassandra for online application use cases, Solr for enterprise search, and Hadoop for batch analytics, we wanted to make sure we had the security bases covered in our platform for each technology. The good news for Hadoop users is that many of the security concerns called out by eWeek are handled in DSE.

For example, eWeek says, “Hadoop also doesn’t support encryption on nodes or on data in transit between nodes”. That’s not true in DSE. Because we use Cassandra for storage vs. HDFS, the transparent data encryption we offer in DSE applies to Hadoop data. Moreover, DSE also supplies client-to-node and node-to-node encryption of data for Hadoop as well as Cassandra and Solr.

eWeek also states, “The distributed nature of Hadoop clusters also renders many traditional backup and recovery methods and policies ineffective. Companies using Hadoop need to replicate, back up and store data in a separate, secured environment.” In the same vein, they state later: “Traditional data security technologies have been built on the concept of protecting a single physical entity (like a database or server), not the uniquely distributed big data computing environments characterized by Hadoop clusters. Traditional security technologies are not effective in this type of distributed, large-scale environment.”

One of the nice things about the Hadoop component of DSE is that automatic redundancy and replication is built in to the platform itself, so all of the goodness of Cassandra – which is architected specifically for distributed, large-scale environments – is inherited on the Hadoop (and Solr) side. This equates into Hadoop data being easily replicated in one location or many; across one datacenter or multiple centers; across one cloud availability zone or several zones. Further, it means no single point of failure or write bottleneck as data can be written to and read in any location.

Backups aren’t hard either as all data is stored in Cassandra column families / tables, so typical snapshot backups and recovery tasks are uniform across a cluster.

So if you’re interested in easily integrating Hadoop batch analytics with your modern line-of-business applications and want to ensure both are secured, you should give DSE 3.0 a try. Download DSE, which is completely free to use without restrictions in development environments (note that production deployments do require a software subscription) and see how it can satisfy both your big data needs and your requirements for security.

Netflix delivers more than one billion video instances each month to its subscribers, and it does so with an architecture that expects failure and is designed from the ground up to handle it.  And what database does the largest cloud app on the planet use to power its media business and keep things running no matter what happens?

Apache Cassandra from DataStax.

ZDNet’s article describes Netflix’s architecture from a high level perspective, but to understand what Netflix does in more detail, you can view Adrian Cockcroft’s (director of architecture for Netflix) presentation from our last conference. In addition, you should review their benchmark tests in the cloud that demonstrate the linear scalability of Cassandra (also see End Point’s benchmarks that compare Cassandra to MongoDB and HBase in the cloud).

Whether it’s performance or continuous availability, Cassandra gives Netflix what it needs. As an example of the latter where database uptime is concerned, when Amazon experienced its much publicized outage last October, Cassandra helped Netflix to never miss a beat: “We configure all our clusters to use a replication factor of three, with each replica located in a different Availability Zone.  This allowed Cassandra to handle the outage remarkably well.  When a single zone became unavailable, we didn’t need to do anything.  Cassandra routed requests around the unavailable zone and when it recovered, the ring was repaired.”

So if you’re wondering if DataStax can tackle an application you’re considering for the cloud, you don’t have to look any further than Netflix for your answer. To understand how DataStax and Apache Cassandra can handle your cloud needs, download our cloud white paper and see the short video below.

Maybe it’s because I’ve been a database geek for so long and have always appreciated advice on where not to step when it comes to trying shiny new technology. Or, maybe it’s because I handled so many database software reviews for magazines in the past that I know one size never fits all, and that the claims trumpeted by software vendors about their technology making all your dreams come true just isn’t legit.

For these reasons and more, whenever I get a chance to talk to a group made up of both business and tech folks, I always try and tell them what I’d want to be told – and that equates to telling them why they may not need the software I’m representing.

This brings me to NoSQL. By all standards of measure, the NoSQL market is booming. Some reports have the NoSQL market growing at an average of 60% per year.  Such information naturally causes IT professionals to wonder if they’re missing something by not implementing NoSQL technology in their environments.

Maybe they are, but maybe they’re not. The $64,000 question is: how do you know?

Having been on the RDBMS side of the fence for so long, and having worked with and watched DataStax customers smartly implement NoSQL technology that actually makes a difference, I think I have a pretty good handle on the why’s and why not’s of NoSQL.

I’d like to invite you to join me for an upcoming webinar I’ve entitled “How to Tell if Your Business Needs NoSQL” where I’ll go over a series of questions that will help you determine whether you can benefit from NoSQL.

No fairies. No pixie dust. No magic. Just some honest scoop that will help you get your head around whether you can use NoSQL right now or not, with clear examples from real customers that demonstrate when NoSQL is actually needed.

DataStax has quickly expanded over the past two years, growing from 26 employees and 27 customers at the end of 2011 to approaching 100 employees and 270+ customers by the end of 2012 along with many thousands of organizations and individuals working with open source Cassandra.

We had some fun with the press this week and they seemed to like the story…

With a strong base of customers in the EMEA region combined with the exceptional demand from the Cassandra community the opening of an EMEA subsidiary headquartered in London to support this demand was a natural next step.

We have also seen strong interest from system integrators, solution providers and consultants in the region and working with this partner ecosystem is a key component of our go-to-market and customer services plan.

Here are some things you can do immediately if you have an interest in Cassandra.

• Cassandra groups have been established in all major cities and you should sign up to the one nearest to you at www.meetup.com. You are going to get a chance to network with your peers and learn how Cassandra can help you with the data driven solutions you are looking to deploy yourself or at one of your customers.
• If you would like to talk directly to the DataStax EMEA team about our commercial offering called DataStax Enterprise, its broad set of features and 24×7 support, please contact us at customersevices@datastax.com.
• If you are a system integrator, consulting firm or solution provider and would like to deliver DataStax Enterprise and Cassandra services and solutions please email the EMEA team at emea-info@datastax.com.
• Certification training will soon to be available in the EMEA region and if you would like to register your interest please also email training@datastax.com.
• We are always on the look out for talent and if you would like to enquire about career opportunities within the EMEA team please send an email to jobs@datastax.com.

It is exciting to see so many established enterprises and young start-ups in the region already relying on Cassandra and DataStax Enterprise for their mission critical data driven applications and services and we look forward to being able to now work closely with them as they continue their journey in this new data driven world.

DataStax: Guys, thanks for making the time to talk with us today. Can you give us a quick overview of Datafiniti?

Datafiniti:  Datafiniti is a search engine for data.  We’ve built a catalog of all structured data available on the Web; things like businesses, people, products, and more.  We keep a massive database built on DSE of all this information that can be searched and used to generate custom output that meets our customer’s inputs and criteria.

We were originally focused on Web crawling technologies at a different company for about three years and then we pivoted that into our new company, which is Datafiniti.  Our customers subscribe to our service and pay us based on the amount of data they receive from us.

DataStax: How do you technically make all that happen?

Datafiniti:  There are two major components to our stack. One is the crawling infrastructure and the other is the search part, all of which is hosted in our data center. Where the crawling part is concerned, we have a number of servers that connect to volunteer computers all over the world that help collect our data for us.

Our search component makes use of DataStax Enterprise with Cassandra and Solr, with an API that sits on top of that for customers to query our database.

DataStax: Did you guys start out using NoSQL for your database or transition from an RDBMS?

Datafiniti:  We didn’t consider relational technology, but started out by looking at all the various NoSQL options like HBase and search software like Elasticsearch.  We ended up deciding on Cassandra for its non-centralized approach and easy scaling. Cassandra allows us to store the big amounts of data that we need to consume and manage.

What was missing in Cassandra was the Google-type search functionality that we needed. When we saw that DataStax had integrated Solr with Cassandra in DataStax Enterprise, it was just a natural evolution for us to use it as our database.

Also, when we tested Solr in DataStax Enterprise, we saw that it worked and performed better than open source Solr, which was also a win for us.

DataStax: Did anything else come into play with your decision making process?

Datafiniti:  At the time, manageability was something important to us – we wanted a database that would be easy to install, manage, and grow. DataStax Enterprise was just the best option for what we do and need.

We also looked at various benchmarks and saw that Cassandra ran faster than the other options we were considering.

DataStax: What are the some of the business benefits you’ve experienced with DataStax Enterprise?

Datafiniti:  The primary benefit is that we’re able to deliver much faster search operations to our customers with DSE, and as everyone knows, customers don’t like to wait long when it comes to searching for what they want.

DataStax: What advice what you give to people who are just starting out with NoSQL and/or DataStax Enterprise?

Datafiniti:  Pay close attention to the demos and samples that ship with DSE because they will help you quickly get things set up and understand how things work. Also, make sure you understand how the individual components of DSE – Cassandra, Hadoop, and Solr – work independently of each other.

Lastly, it’s good to know up front how you go from Cassandra, which has a very flexible and fluid schema model to one that’s more restrictive like Solr.

DataStax: Guys, thanks for the time.

Datafiniti:  You bet.

For more information on Datafiniti, please visit: http://datafiniti.net/.

DataStax: We’re doing this interview live at our Cassandra New York Big Data event. Stephen, what are your impressions of the show so far?

Stephen:  It’s been great. I’ve gotten to meet with a lot of unique representatives from some major brands like eBay and others. I’ve learned a lot from the folks over on the front lines working with Cassandra directly and I’ve had a chance to send some ideas across and get some very helpful feedback.

DataStax: What does see.me do?

Stephen:  See.me is an online community for creatives. So whether you’re a photographer, model, musician or designer you can join our community and support other creative, be supportive and pretty soon earn money from your passions.

DataStax: So let’s talk a little about Cassandra, I understand you have a MySQL background; why the switch?

Stephen:  At see.me we’re being proactive about the needs of our users and future user base, which is growing. Right now we’re about around 700,000 users and growing a few thousand a day. We foresee that MySQL can’t handle our data ingestion needs and it also doesn’t address things like search and analytics.

But I would say the biggest motivator for the switch is that Cassandra scales readily and is designed for scale. We need it for resilience, stability and to handle the growth.

DataStax: You mentioned search and analytics, so I understand you’re interested in Datastax Enterprise and just doing some research and getting to grips with it. What are some of the features that you find attractive in that platform?

Stephen:  For Cassandra itself, I initially did the research for a different company years back and I was interested in the scalability and performance for reads and writes. Specifically, with DataStax Enterprise, I’d say that we really love the idea of an out-of-the-box solution that combines, without ETL, Cassandra for storage, Solr for search (which we’re already using with MySQL to great effect), and also Hadoop for analytics. And we have work to do on designing our algorithms and getting smarter for not just matching up creators, but also with images and aesthetics. Bringing this all together, being a small business without a big budget, we can do a lot for our users with DSE.

DataStax: What advice do you have to pass on to people who come from a relational background and are getting started with NoSQL and DataStax Enterprise?

Stephen:  I would say take advantage of the fact that Cassandra has, from very early on, been very open sourced and community engaged.  There are lots of resources including webinars and videos that convey to you, faster than reading the manual in some cases, how you can use this to get up and running quickly and that was important to me.  Participating in a webinar allowed me to understand not just which version of Cassandra was right for my company, but which version of Cassandra I needed to sell to my teammates to make the transition from MySQL and to really sell the importance/impact of it for people who are not on the front lines of data.

DataStax: Steve, thank you very much for taking a few minutes today at NYC* Big Data Tech Day to talk to us. Enjoy the rest of the show and we look forward to having your participation in the community moving forward!

Stephen:  Thanks!

For more information on see.me, visit: http://www.see.me/