Automating data aggregation for a multinational FMCG brand
To save time, resources and money
Tech Stack
- Java
- MySQL
- Spring Boot
- UIPath
Expertise
- BI Analytics
- Business Process Automation
The background
Unilever is a global manufacturer of fast-moving consumer goods (FMCG), including food, beverages, beauty products and cleaning supplies, operating in more than 190 countries. For a multinational company of this size and scale, raw material sourcing across its global supply chain is a huge, ongoing challenge.
The challenge
To gather the data in a more efficient way, Unilever needed to automate the data scraping process from 70 websites, which share information about important market trends. Each website had to be crawled at a different frequency too, ranging from twice a day to once a month, depending on the significance and volatility of the data.
As well as monitoring raw materials, Unilever also needs to consistently source construction materials to build additional factory units to keep up with demand for its FMCG items. Yet understanding the ever-changing cost of these supplies, which can be impacted by factors including production costs, global supply and demand, forex rates, logistics, inflation, commodity rates and fuel rates, proved difficult.
But as the data formats and the URLs were constantly changing on many of the websites, it was difficult to automate the process and ensure it extracted the right information every time. To make matters more complex, lots of the information needed to be downloaded as a file before the information could be analyzed by any automated tool and extracted.
Our client had a dedicated team who manually aggregated data to keep decision makers informed about factors that could change the price of materials. But this manual process was labor-intensive and vulnerable to human error, which resulted in poor decision making that was impacting Unilever’s bottom line.
The solution
To meet our client’s requirements and timeline, we started by forming two separate teams that would work in parallel on different aspects of the application. Team A focused on the automation script/bot, which would scrape relevant data and send it to the application’s backend. Simultaneously, Group B developed the backend service, application programming interfaces (APIs) and the web-based dashboard using Java and Spring framework.
Following our recommendation, Unilever chose UiPath as its automation platform, which would extract and parse data from its chosen websites. We chose to develop the frontend using Angular JS, with Angular Material and Bootstrap components. This gave the single-page application (SPA) a contemporary appearance and impressive functions. We also used Chart.js, a reliable Open Source library, to render data into eight to 10 different types of chart that would suit a variety of different types of data.