At NITS Solutions, we’ve been storing all our data in Oracle from day one, and for many years it did a great job. As our business continued to grow however, Oracle started to creak and groan under the weight of so much data processing. Pending jobs were piling up as new products, re-summarizations, and analyst queries continued to increase. We had to find a way to lessen the demands on our database by offloading a lot of tasks to a new system. This is what spurred our investigation into implementing data lake technologies.

Goals

Our goal is not to entirely…


For those of you just getting started with building your own data lake, there can be a dizzying array of technologies and processes at play. For our team at NITS Solutions, understanding all of the nuances has certainly been a steep learning curve. Each build depends on the problem it is trying to solve, but luckily, many solutions can be distilled down to just a few key points. If you’ve already read Part I: Is a Data Lake Right for Me?


If you’re anything like me, your first exposure to the world of data science probably started with a single SQL query. With good reason too, as relational databases provide a very intuitive, user-friendly way to interact with large amounts of data. But what happens when your business matures, the problems become more complex, and your relational databases can’t keep up? We’ve been experiencing these growing pains at NITS Solutions lately and have finally settled on building a data lake as our preferred solution.

What is a Data Lake?

Interest in data lakes has grown quite a bit in the past…

Stefan Lopez

Data Team Manager at NITS Solutions. When I’m not solving tech problems you can probably find me on the disc golf course. https://stefanlopez.tech

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store