Anishagarwal (1)

1
BEHIND THE SCENES OF A MARKET
INTELLIGENCE PLATFORM
Is there a way to capture intelligence and make
it available to the right audience, at the time
when it is needed? How can intelligence on
competitors strategies best be gathered with a
market intelligence platform? Here are some of
the best practices based on how leading firms run
their market intelligence system.
2
CHALLENGES IN DEVELOPING A MARKET INTELLIGENCE
PLATFORM
Identifying companies and tagging
Sourcing of information
Industry tagging
Removing irrelevant information
Company tags and mentions
Social media tagging
3
SOURCING OF INFORMATION
The platform has to integrate with thousands of
different websites and continuously monitor
those websites to detect new information. Some
challenges a sourcing engine faces are Most
websites do not have RSS feeds and APIs
available. There are no universal standards for
website development. Web scraping - This is more
like a dark art which requires specialized
skills.
4
REMOVING IRRELEVANT INFORMATION
You can remove the non-business information right
at the source. For example you can remove
stories with the word kill but be careful,
don't remove the stories like Google aims to
kill passwords.. You have to remove information
that is related to business but not relevant to
our business. For example, information about
your industry but from a different geography, or
information about your competitor but for a
different segment where you dont compete.
5
REMOVING DUPLICATE OR SIMILAR INFORMATION
Group information based on standard algorithms,
then ungroup based on the other signals in the
article, such as the industry, the topic, the
companies etc. The accuracy improves with each
step of grouping and ungrouping. After
successfully grouping, you will realize that a
less important article is at the top of the
group. So now you need to somehow make the
algorithms understand which is the most
important article in the group.
6
IDENTIFYING COMPANIES AND PERSONS
There are common English words that are bane to
the dark world of text analytics like
There are company names which are common nouns
or popular words, such as Apple, Amazon,
Gap. All this makes even more difficult for a
machine to accurately recognize entities in an
article.
Look for words that have the first letter in
uppercase. For example, ICICI. But not always
it turns out right like for instance,ICICI Bank
Q3 There are misspelt words and fancy foreign
language words such as LOréal that add to the
misery.
7
IS IT ABOUT THE COMPANY OR JUST A MENTION?
How do we know whether the story is about the
company or just mentions the company? One way
to address this problem is to assign scores to
all the companies in an article. But it's easier
said than done. It is not a good idea to start
developing and maintaining a knowledge base for
each signal. You will need many as you aim for
higher accuracy. Rather, you should find
partners who can feed us such data for
signals. Fortunately, many companies provide
data feeds, which can be used as signals, via
APIs.
8
INDUSTRY AND TOPICS OF THE ARTICLE
You need to fine-tune the classification
algorithms to recognize the pattern of words
which are commonly used to describe an industry
or topic. Even in this, as you go deeper, you
find increasing complexity. You will be damned
if you tag a story incorrectly, and damned if
you miss the tag. You also need to preserve the
relationships between the topic and the
companies.
9
HOW ABOUT SOCIAL MEDIA?
To extract a few relevant pieces of information
from the millions of mindless shares and updates
is very difficult. There is an increasing number
of social media sites with increasing
complexity. It is not easy to even find the right
social handles to monitor. There are fake
handles, multiple different handles of people,
and companies for different purposes.
10
CONCLUSION

• If these challenges are mitigated then these
  platforms will not give you great platforms even
  if you have great technology team
• Example Uber with over USD 8 billion in funding,
  uses Stripe for payment processing, and Twilio
  for SMS messaging. You should think hard before
  deciding to build a market intelligence platform!
• If you are someone who depends on market
  intelligence or youre responsible for running
  the market intelligence program, do take Contify
  for a spin.

11
Learn more about market intelligence platform
Take a 7-day Free Trial of Market Intelligence
Platform Try Now
Read more at https//www.contify.com/blog/behind-
the- scenes-of-a-market-and-competitive-intellige
nce-platform/