Our client had a large volume of data from various sources and formats, including APIs, XML, TXT, and CSV files. The data was non-standardized and contained a large number of variables and constants, making it difficult to use and analyze. The data also arrived on various schedules, sometimes without any schedule at all, and contained millions of strings that sometimes overlapped with other datasets but had different parameters such as quantity, ID, price, and arrival date.
To help our client overcome these challenges, we developed a custom web-based product with dedicated portals for different user types, including buyers, managing partners, and internal managers. The product included a range of features to cover multiple use cases, as well as a data processing engine to provide a single output for many data sources. We also developed a custom matching algorithm to identify potential duplicates or matches across the various datasets.
The product was built using technologies such as
— Java, Elasticsearch
— Docker
— Phalcon
— Material UI
— Styled Components
— Redis
— PostGres
— Gearman
— RabbitMQ
Project was managed using a hybrid-agile approach.
Our client approached us to create a web-based product that would allow them to quickly search through this data by price, arrival date, quantity, and substitutes for original goods. The product needed to serve multiple user types, including buyers, managing partners, and back office managers.
We were able to create a multilingual cross-domain web product with dedicated portals for each user type. The product included a range of features to cover multiple use cases and a data processing engine to provide a single output for many data sources. This helped our client to overcome data processing and customer processing issues that could have blocked their expansion. In the first year of use, the client saw a more than 200% increase in turnover compared to previous years.
By developing a custom solution that addressed the unique needs of our client, we were able to help them overcome their data processing challenges and drive significant business growth. If you are facing similar issues with data from multiple sources and formats, we can help you find a solution to improve your business performance. Contact us to learn more.