When data management comes up, people normally think of databases. However, many additional solutions are required for the complete strategic management of data. This article will go over the concept of the data lifecycle and all the tools that go with it. From the collection phase right through to archiving, find out how to optimize data management using technology.
Quick reminder
In our previous article How to Get Started with Data Management, we shared multiple tips on how to properly manage data within your company. We also went over the data lifecycle. This cycle can help you to understand how data flows through your company. Drawing the data lifecycle diagram can help you to identify data streams and what tools are required to manage them.
This article will go over all the digital tools that can help with each stage of the data lifecycle. However, we recommend that you read our previous article first to understand a few strategic issues, such as how to choose the type of data your company should collect. That way, you’ll be able to quickly see what tools apply to your situation.
Tools for every stage
The data lifecycle consists of seven stages, not including final deletion. These stages can be grouped into three main categories:
- Gaining knowledge (creating)
- Using knowledge to make decisions (processing, analyzing, sharing, reusing)
- Preserving knowledge (storing, archiving)
As data accumulation can sometimes increase exponentially, some companies also consider permanently deleting certain data. However, this decision should not be made lightly. If you ever find yourself in this situation, take a look at these tips from LesEchos:
- Assess the risks associated with your data to minimize their impact
- Learn more about your data and how it flows through your business
- Focus on assessing the value of sensitive data
- Make sure your data storage is secure
- Use automation for smart data deletion
- Apply the principle of least privilege to better control access to data.
After reflecting on these issues, you’ll be able to identify which pieces of data have no real strategic use and are simply wasting your valuable storage space.
However, before deleting your data, it’s important to use it to the greatest possible extent. To get you started, here are the most useful tools out there.
Tools for collecting data
Your company most certainly has a lot of data already. The tools you currently use are what generates your internal data. They provide you with a lot of important information, including your turnover and customer retention rate. However, they are not always sufficient. You should also consider collecting external data.
There are many sources of data that you can use to expand your knowledge. Some data is considered open data, including that from public institutions. This type of data can help you to gain a better understanding of your business environment. However, you can also take things a step further and collect data on your own through research. Here are some types of data that you may find useful:
- Data on internet users and customers
- Data from sensors and connected objects
- Data on suppliers, B2B clients and competitors
- Data on markets
- Geographical data
- Other data, including images
To collect some of this information, you can create surveys using tools such as Qualtrics and SurveyMonkey. Surveys are especially useful for finding out more about your consumers and their behaviour. You can also use the well-known NPS survey to evaluate your service quality.
If you have a website, you should also use data analytics tools, such as Google Analytics. These tools can help you to gain a better understanding of user behaviour on your platform so you can make improvements on an ongoing basis. Your social media platforms can also provide you with a lot of data. Facebook, Instagram, Twitter and LinkedIn all feature sections that provide key statistics.
If you would rather analyze your business environment and market on a macro level, you can also look at research from other organizations, such as Euromonitor, MarketLine and Gartner. However, access to these databases comes at a significant cost that may not always be justified depending on your context. Reports from Cefrio or Light Marketing can be excellent alternatives. And don’t forget that you can always use search engines to bring up relevant results!
Another alternative can be to use qualitative data collection methods, which are often overlooked without good reason. Focus groups or individual interviews with your customers and prospects are one option that can help you take some types of strategic thinking further.
Tools for storing data
Enterprise Resource Planning (ERP) software
Both internal data and external data need to be stored somewhere once collected. Many enterprise software applications provide for this, the most well known being ERP or Enterprise Resource Planning software. This business management software makes it possible to store data from various departments in a central location in order to (most commonly) engage in strategic planning. More specifically, ERP software makes it possible to:
- Enter a piece of data and instantly find it in all of your systems
- Optimize your departments and the way they work
- Connect all departments together to automate activity flows
- Manage your business in a centralized way from a single database
Customer Relationship Management (CRM) system
The other tool that is most commonly used is a Customer Relationship Management system or CRM. A CRM is a system that can be used to handle all of your sales and marketing activities. A CRM can be used to:
- Store customer information
- Employ tactics to acquire new customers
- Automate tasks previously handled by your sales team
- Nurture existing relationships with your customers
- Save information in your database
- Analyze data
There are hundred of CRMs, including Salesforce, Microsoft Dynamics CRM, HubSpot and Zoho. Your choice will depend on your business context, resources and how important your data is. There are actually several categories of databases to choose from. Relational databases are definitely the most well-known for everyday use. In these databases, information is categorized and defined in such a way as to be easily accessible. These databases are accessed using SQL (Structured Query Language) and are especially useful for companies that are getting started with data management or that don’t have excessive amounts of data to process.
If you have an excessively large volume of data, other solutions may be better suited to assist you. And by other solutions, we mean non-relational databases or NoSQL. These databases store information in key-value pairs, making it possible to accumulate more data. Once again, there are several types of non-relational databases which may be more or less suited to your context. If you prefer using open-source technology, you may want to try MongoDB, Cassandra, Hypertable or Couchbase.
Database management systems and data warehouses
In addition, to work with databases, you need a database management system (DBMS). This type of software allows you to store, view, update, structure or share the information contained in a database. It can also help you to manage your databases once you have several enterprise software applications. One of the systems that is most commonly used is the data warehouse. This system often includes a database, an extract, transform and load (ETL) solution, statistical analysis capabilities and other analytics applications.
Now that your data has been stored somewhere, you need to be able to easily use it and add to it on a daily basis. However, daily management can become difficult once you have several enterprise software applications. To make sure that the same tools can be used to meet the needs of several departments, companies are increasingly using interface solutions or connectors, such as APIs. They make data management easier while giving the right people access to all of the information, which is also presented in an optimal way.
It’s true that it might not be easy to choose the best infrastructure for your needs. That’s why it’s preferable to go over your options with experts in the field.
Data processing and analysis
Processing data means performing certain operations that make the data comprehensible and usable. However, you can’t make meaningful calculations or perform relevant analyses on your data if it hasn’t been prepared. The preparation phase is often carried out by data analysts or scientists. During this phase, their role is to clean up the data that is contained in your databases, meaning that they delete the erroneous data and make sure that there are no outliers so that the information can be used for calculations.
Dashboards
Once your data has been prepared, it’s ready to be processed. You can then run a series of calculations and algorithms on it depending on your needs. Your data can also be processed by a tool, such as a dashboard. Dashboards provide a visual representation of several key indicators and metrics. They can be of great use when making decisions.
To be useful on a daily basis, a good dashboard should help you to:
- Grasp a context or situation by presenting 2-3 key pieces of information
- Effectively and easily share information with your teams
- Be more responsive due to instant data
- View data from several sources in 1-2 relevant graphs
- Find out what you really need to know in one glance
Statistical tools and business intelligence tools
There are several interesting tools available in this category. For a general analysis of your data, solutions such as Tableau Desktop, IBM Business Intelligence, SAP Analytics Cloud or Power BI can be useful. If you want to take your statistical analysis further, you can try out tools such as SAS, SPSS or R. If you’re a beginner, we recommend R—an open-source solution with a large community.
You can find a longer list of analytics tools here.
Web and marketing analytics tools
To analyze data from your web platforms (your website and/or application), you can use the various tools provided by Google Marketing Platform or take things further by using solutions such as SEMrush or Moz Pro. These tools can be used to both collect and analyze data in real time.
You can find a longer list of tools for digital marketing here.
To analyze data related to your social media platforms, you can use their own analytics solutions, including Facebook Insights, Twitter Analytics, Pinterest Analytics and Instagram Insights. In fact, most platforms that generate data provide tools to help you analyze and monitor information on a daily basis.
You can find a longer list of tools for social media here.
No matter what tools you use, the important thing is to choose the ones that truly meet the needs of your business and employees. That’s why nothing beats a customized analytics solution!
Data archiving
Earlier, we discussed storing your data. Although storing data is often confused with archiving, the two concepts are distinct. Archiving means transferring data to another system in order to gain storage space and prevent tools from becoming overloaded. It’s more than just backing up your data by saving it.
There are several open-source solutions that can help you to properly archive your data, including Alfresco and Odoo Documents. You can also push the envelope by installing electronic discovery (eDiscovery) software, such as Disco, to make searching in your archives easier.
Once again, it’s important to think strategically about what you want to archive. Check out some tips here.
Data sharing
You may also want to share your data—that’s normal! Of course, you can do this using your enterprise software, which stores your data in a central location and distributes it to the right people. However, there are also other well-known data sharing solutions, such as WeTransfer, SendSpace and Dropbox.
Data sharing also comes with many security considerations. VPNs can be of great use if you want to provide your teams with access to your software and data while they are outside of the office. NordVPN, Perimeter 81 and ExpressVPN are simple solutions that your company can implement. However, make sure that you select the solutions for businesses, not the ones for individuals.
When you start sharing your data, it’s important to assess and oversee data security, especially if your data includes personal information. This issue should not be taken lightly. Check out our article on data governance to learn some good data security habits.
Reusing data
Reusing your data is a wise move. That way, your efforts to collect it can continue paying off over the long run. If you want to reuse your data, you should make sure that your solutions are optimal so that searching for information is as easy as possible. Indexing your data and using internal search engines are good strategies.
There are also great open-source solutions for this stage in the data lifecycle, including our favourite—Elasticsearch. In addition to making it easier to search for information, Elasticsearch also comes with an interface called Kibana, which has smart features that help you visualize your data. We put this technology to use as we carried out the Panorama project for Radio-Canada’s Hackathon.
Technology tools are part of data governance
Yikes! You might be a bit apprehensive after reading the previous sections of this article. There are so many solutions; however, not all of them apply to your business context. The most important thing is to choose solutions that meet the needs of your business and users.
Strategic reflection around data management tools should take place during the development of your data governance policy. In a previous article, we explained that data governance is made up of four pillars, one of which is technology. To make decisions about the technology pillar, you should definitely get input from experts, which brings us to our next piece of advice for you—don’t dive straight into selecting technology tools without first engaging in a technical and strategic analysis with professionals.
Points to remember
We aimed to make this article as comprehensive as possible to give you an overview of all the technological considerations that come with data management. The goal was not to scare you off, but rather to drive home the point that data management requires a lot of strategic thinking.
Your data has a lifecycle. If you understand the stages, you can use digital tools to optimize all of them. And that can offer you a world of strategic opportunities— don’t miss out!
test
