.png)
Your company’s data is one of its most valuable assets. While the promise of automated machine learning and data mining seem attractive, your company will still need people to help you generate value from data for the foreseeable future.
One of the questions we get frequently is: why should I partner with an independent data team rather than hiring one internally? Before diving into the reasons for bringing in an external or internal data team, it’s important to settle on the definition of “data work” that your company needs to be completed. In CorrDyn’s Taxonomy of Data Work, we establish twenty distinct Skill Sets that can be incorporated into a given data project:
These Skill Sets are distributed among different disciplines within the data world, such as Data Engineering, Data Science, Business Intelligence, Machine Learning Operations, and Data Architecture. Generally speaking, every discipline is a bucket of Skill Sets, and these disciplines overlap in terms of the Skill Sets they can provide. Follow the link above for a deeper description of each Skill Set and the disciplines that encompass them.
The argument for an external analytics team is strong, which is why our team believes that we can consistently generate value for our clients. Below are the ten most important reasons for hiring an external data team:
Good data partners do the work and get out. Full-time data teams have to justify their value. Data projects usually require 2-12 months of intense development time, followed by a slower stream of enhancements and an even slower stream of maintenance tasks.
If your project can be reliably automated with minimal ongoing maintenance, then you do not need a full-time data team to manage it. If you hire a full-time data team for projects like this, that team does not have an incentive to automate themselves out of a job. A good data partner automates themselves out of the current project to make room for the next project.
Depending on the nature of your company’s data projects, you may require a different allocation of time and skills for each project. If your projects are not big enough to justify a full-time data team or even a full-time data engineer, data scientist, or data architect, then you will save money by bringing in a team that can allocate the right resources with the right expertise on an hourly or project basis.
Even if your company has a sufficient set of data projects such as machine learning products or business intelligence reports that do require a full-time team to develop and manage, data scientists and business intelligence analysts need sufficient data infrastructure to meet your company’s needs in a timely manner.
If your company’s data has not been adequately piped, integrated, and cleaned, then your analysts and scientists will spend the vast majority of their time on those tasks. If you hire a data scientist or analyst to clean data, that person will either cut corners to deliver their product or spend their discretionary time looking for another job.
You may have a data person or even a data team, but your company needs a fast prototype of a data product that you do not have the capacity to develop with the degree of speed or quality your company requires. A good data partner can help you understand the scope, prioritize functionality, and deliver a minimum viable product on time and within budget.
An internal data team is expensive, with most qualified team members commanding a six figure salary to start. Your company can get high quality work and share the expense with other companies by only asking your data partner to complete the projects your company needs. For the cost of one FTE, you get all of the expertise of a team, each devoted only to the tasks for which they are most qualified.
A data engineer can’t understand your business and deliver the reports you need, and a business intelligence analyst can’t build the machine learning models you need, and a data scientist can’t always build the customer-facing data product you need. As the skills your company requires expands, an external data partner can extend the capacity of your internal team to deliver the outcomes that your business requires from its data.
We often see companies who hire one “data person” to get all the data work done. With the breadth of skills and depth of understanding necessary to build quality data products, this approach usually results in cutting corners, and cutting corners creates rework later. If your company wants something to build on, bring in an external data partner to develop the infrastructure for your “data person” to be successful. When that data person leaves, you will be thankful.
Related to #5 Data Team Members are Expensive, you should only hire for the disciplines that truly need FTEs at your company. For other disciplines that only need 20-40 hours per month or a limited full-time engagement, your company needs an extended team you can call on to deliver high quality work without added administrative burden. Major software companies like Microsoft, companies that have plenty of data expertise internally, leverage this model to ensure that they can deliver on the outcomes their business needs without ballooning their staff unnecessarily. What works for them can work for you.
Software is eating the world, and where there is software, data follows. If data expertise is not how your company competes, then an external data partner can enable you to focus your attention and resources on the areas where your company does compete. The external data partner can bring in the expertise and resources needed to accomplish your goals, deliver high quality products, and lay the foundation for a future in which your company begins to compete on data expertise.
Unlike your web development and application development teams, data projects normally do not interact with hundreds of combinations of browsers and operating systems and legislative regimes. Data pipelines can generally be automated more reliably than customer-facing software development because it changes less frequently and more predictably. When a data pipeline does break down, that breakdown is often less urgent than a website or application failure. This means that your service level agreements can be supported by an external team, at lower expense than a full-time team.
With that said, many companies do need to hire their internal data teams. The four reasons below are the most important reasons why you should insource your data work:
If your company has enough work for multiple FTEs, then hiring the internal team makes sense. Your company should build repeatable data practices internally. Your company should make it easy for new hires to build on the work of previous employees and develop internal systems and knowledge sharing practices for people to quickly deliver value to your business. An external team will always have to spend time understanding your business context and will have to build momentum over time to deliver value to your business.
If your company already has strong data leadership, then you are in a position to set your internal data team up for success. Your team will receive the guidance, resources, and support they need to deliver value. They will not have to spend time figuring out where to deliver value before they start work. The team will receive clear direction and strong opportunities for professional development. Your team can hire relatively junior data people and grow them into strong contributors.
If your company needs to lead in data, then an external team will only get you so far. Your company requires strong internal expertise and incentives that encourage your data team to expand on the value they deliver. If your data team does not deliver, that is a critical failure for your company. You need the right data people internally to become the company you need to become.
Related to #3 Data is a core competency or competitive advantage, if your company’s product or service is based on data, and that platform or service requires 99.999% availability, then you need a full-time team to operate and enhance that platform or service. An external data team can probably never have the level of responsiveness you would require from an internal data team.
At CorrDyn, we provide an external data team that can deliver value to your business, whether as a baseline for your future data competencies, complement to your internal data team, a lifesaver for your “data person”, or a strong reason not to invest in an internal data team at all.
