Mass merchandisers, like Walmart, generate data in the order of TBs per hour, which needs to be retrieved, stored, and then processed to be made available to our clients for real-time analytics. And, all this has to be done before the useful life of the data expires, which again, for retail data, is a matter of a few days.
Timestar’s engagement usually involves identifying retailer data to import and store, and we boast experience with a multitude of big and small retailers. In addition, we can process POS/Inventory data direct from stores, to bring mid-, to small-segment retailers an in-house DB with an interactive analytics capability, in addition to other customer analytics modules processed on a Hadoop cluster, using MapReduce.
To store this huge amount of data, we utilize multi-node servers connected through ultra high-speed Gigabit ethernet, running 24/7 to process, and serve the requirements of our clients. Our servers are placed in an SSAE16 audited data center that lies in a lead-lined bunker with six layers of secured entry. The security, and constant monitoring of the physical location, is only matched by the constant network/service monitoring of our servers by our NOC/SOC partners, and they alert us to any service breach, by monitoring application logs, or to lack of service availability, due to a server breakdown, within a minute of the event. Again, through our partners, we have a “less than 4 hour” service restore guarantee, and we have never, in our 15 years of serving our clients, breached that agreement.
Our servers run a high-performance, in-memory, columnar database, tuned to provide interactive analytics and discovery. The DB stores multiple TBs of data, available live for our clients’ use. In addition, we keep an archive of historical data that is no longer required for interactive insights, also live at all times, and ready to be made available to our clients at a moment’s notice, in an off-chance case of a historical analysis. Both, the available and the archive databases, are backed-up regularly to an external NAS, and is subsequently moved to an off-site location for preservation.
Read More >>>
We pride ourselves with staying on the cutting edge of data science technologies. We were among the first 30 customers of Vertica, which at that time was the only in-memory columnar database available in the market. We have been working with them since then, and our relationship, and easy accessibility to the top engineers there help us fine-tune our DB platform, in addition to understanding more about the capabilities that Vertica has added in it software over the years, using it to do time-series analyses for our clients. This extends our value-add whereby we bring additional insights to the retail DB that our clients cannot get directly from the retailer. Having these insights, in addition to sales metrics, readily available in a high performance DB brings our clients the capability to identify issues sooner than their competitors, and take responsive action.
We also have a strong relationship with Tableau Software, even helping them with presentations during their roadshow to market their software to big retailers. Our expertise with Tableau has helped us create stunning visualizations that are used by our clients hundreds of times a week. Our clients love these dashboards, and snapshots are regularly used in presentations to their retailers.
Our highly skilled engineers have experience with PHP (Cake/Zend) and client-side jQuery/AJAX for web development, and we have created multiple HTML5 dashboards for various screen sizes, including tablet-friendly dashboards that our clients can take along on their iPads, etc.
We are currently developing modules using the MapReduce model, deploying applications written in Scala, on an HDFS cluster. We are working with our clients to get them predictive capabilities, using R, helping them budget sales/inventory at a store/item/week level.