When Manual Research Failed: How We Built Our Own Market Insight System

Overview

Built internally by the product and marketing team after the company missed two competitor feature launches within one quarter. Weekly reports previously took two team members nearly three full days and still depended on screenshots, links and outdated spreadsheets. To solve this, the team built an internal AI platform that tracks competitors, surfaces trend signals and produces weekly insight reports. Reporting time has been reduced by more than half, and the system is now used by 11 people across marketing, product and leadership.

Background

Becky works at a small startup that builds an AI writing product used in offline workshops and coaching sessions. The product has around 1,000 users. As the user base grew, the team needed a clearer understanding of competitor activity, emerging standard features and which AI trends were gaining attention. The team initially tried to track this manually. Notion pages, Slack messages and browser tabs were filled with half-saved links. When someone asked, “What did our competitors release this week?”, it took hours to answer. More than once, they only learned about competitor updates from users. That was when it became a real problem.

The Turning Point

In April 2025, the team missed a competitor release for the second time in three months. The announcement was already being shared in customer communities while the team was still unaware. In that meeting, they agreed that manually collecting information was no longer sustainable. They did not intend to build a product. They only wanted a system that made sure they would never walk into Monday meetings unprepared.

What They Built

They started small, with a basic system that could: • Monitor competitor websites, product pages, Twitter posts and Product Hunt listings • Detect frequently appearing keywords and topics in AI writing and education tools • Generate a weekly insight report the team could review before meetings It was never built to predict markets or replace analysts. It only needed to gather, organize and summarise information faster than a person with fourteen open tabs.

What It Looks Like Today

Competitor tracking

• Tracks 7 direct competitors • Monitors around 15 sources, including websites, press pages, job boards and Twitter • New updates are stored with links and timestamps, so nothing relies on memory anymore

Trend and keyword monitoring

The system identifies which topics are being discussed more often across public articles and announcements. It shows whether interest is rising or falling, instead of listing static keywords.

Weekly insight reports

• Generated every Monday morning • Includes key updates, feature changes, trend notes and a short AI-written summary • Reports can be exported as PDF or slides when needed

AI assistant inside the dashboard

Team members can ask questions such as “What changed in pricing last week?” or “Show updates from company X.” Answers are based only on data already in the system.

What Changed for Them

Before:

• Two people spent about three working days each week collecting information • Reports were delayed and often outdated by the time they were shared • Important updates were still missed

Now:

• Reports take less than one and a half days to prepare • Eleven people across marketing, product and leadership use the dashboard weekly • Monday meetings focus on decisions rather than searching for information • The team reacts faster to competitor activity because they see it earlier This tool did not generate direct revenue, but it changed how the team works. It removed guesswork from discussions and helped them shift from reacting late to preparing early.

What Still Isn’t Perfect

The system is not flawless. It sometimes pulls irrelevant content when keywords appear in the wrong context, and AI summaries can feel generic when the original source lacks depth. Some competitor websites block automated access, so important updates are still checked manually. The interface was designed for internal use, making it practical but not polished. The team relies on it every week, but it remains a work in progress.