Relevant data for pricing

While pricing strategies may vary a great deal from one business to another, what we have observed is that good pricing strategies almost always require a substantial amount of data. Nearly always, most pricing tools tend to place a major deal of importance on the prices of competitors. At Lokad, although we recognize that competitor prices are important, we observe that it is also frequently the tree that hides the forest. Competitor prices do not tell you whether it is a good option to undercut or to align your prices vis-à-vis to your competition, and more importantly, they do not tell you which products are key for doing so, or where and when this should be done.

Historical business data

When it comes to pricing optimization, many merchants miss the obvious: analysis should begin with the “core” data of the business itself. Indeed, this data is, by far, the most reliable and the most relevant source of information for any business. The Internet is vast, and every single product you sell is probably sold by dozens of other businesses around the world. However, depending on your industry vertical, clients may not systematically spend time searching the web to uncover the least expensive option; at least not for all products and not for every single purchase they make.

Granted, certain verticals like consumer electronics are extremely competitive, but even for those rather extreme situations, we observe that while extreme competitiveness applies to flagship products, accessories and consumables enjoy much more comfortable profit margins. Running a retail business costs money, and hence money has to be made somewhere. By decreasing order of importance, your core business data includes:

• list of products sold and their attributes,
• inventory stock levels, and incoming purchase orders,
• list of past items sold,
• list of past purchase orders,
• list of past cancelations / returns / chargebacks / incidents
• list of past prices on display
• web traffic aggregated per product per day
• SEM (Search Engine Marketing) costs per product per day
  Among all this data, the last two points are typically the more difficult ones to consolidate if only because the volume of relevant data is typically 100x larger than all the other datasets put together. However, except for these last two points, retrieving the mentioned core business data should take precedence over acquiring data about your competitors.

Lokad provides native support for many popular [commerce management software connectors] (/platform/integrations/) such as Brightpearl, Linnworks, QuickBooks, TradeGecko, Unleashed and Vend to name just a few. If your app is supported by us already, then importing the bulk of your historical data into Lokad can be done in just a few clicks.

If your app is not supported, Lokad also offers the possibility to upload tabular data files such as Excel sheets or flat text files like CSV. Indeed, every Lokad account comes with a file-hosting service where files can be imported through web uploads or alternative protocols like FTP or SFTP. Don’t hesitate to contact us for more guidance.

Obtaining competitors’ prices

Once core business data is gathered and leveraged for pricing optimization, then, the next step involves keeping an eye on your competition. From a software perspective, monitoring competitor prices involves three distinct steps:

• crawling: the process of automatically navigating through all the links of a website of your competitor in order to discover all the products being sold.
• feature extraction: the process of automatically turning unstructured web pages into a structured dataset, isolating product names, product attributes and product prices (among others).
• product matching: the process of automatically creating pairs of comparable products between products sold by your company and the products sold by your competitor.

Usually, “product matching” tends to be the most complex operation, especially for verticals where there are no “straightforward” product matches, such as fashion.

To date, Lokad does not integrate a competitive price monitoring feature. Therefore, in order to create pricing strategies that leverage the prices of your competition using Lokad, your company needs to provide us with its own data. In practice, most of our clients obtain this data from one of the many competitive price monitoring solutions that can be found on the Internet. Nearly all such solutions provide flat text file exports (eg. CSV files) of the pricing data, which perfectly fits with Lokad’s data processing capabilities.

If your business has a strong IT team, in this case you can even roll out your own solution, with open source tools like Scrapy.org which gives you a very strong head start when it comes to achieving results within a few days.

A cautionary word about competitor data

As long as your pricing processes rely on your core business data, the quality of this data is typically very high. Once in a while, there is an (almost) inevitable incorrect piece of data entry creeping in, but otherwise, data can be assumed to be nearly 100% correct.

In contrast, we observe that even in the best of situations, the quality of your competitors’ data tends to be much lower. Every single step outlined above can fail: you may omit a specific competitor product, an incorrect price might be extracted or a product may be incorrectly matched. In practice, a lot of effort needs to be invested in accurately assessing the quality of this data, otherwise your company is likely to face serious garbage in, gargage out problems.

There are many (many, many…) vendors who claim to have the best competitive intelligence solution on the Internet. As a rule of thumb, we advise to carry out a simple test to put such solutions to the challenge: just ask your competitive intelligence provider for a side-by-side comparison of your web store with the web store of your closest competitor. Indeed, by doing so, you will be able to assess the quality of the competitive intelligence technology by comparing the figures freshly scrapped from the web with the figures you have extracted from your own systems.

The simplest way to identify software vendors who suffer from weak competitive monitoring technology is the lack of a free trial. If no free trial is offered, then, it means that the solution is desperately lacking automation. At Lokad, we have witnessed many times over that merchants are be better served by in-house web scrapping solutions as opposed to opting for weak technology vendors.

Another strong hint of weak technology is when your vendor requests an extract of your product database. In fact, the vendor is asking you for this data to make sure that the data exhibited at the end of the pipeline does not mismatch the data of your own product database. However, by doing this, the vendor also prevents any attempt – on your side – to pinpoint any major data quality problems. In reality, the vendor knows that your capabilities to investigate your competitors’ price figures are very limited, which is not the case of figures obtained from your website that you can directly match to your databases.