AI and Data Case Study: The Database That Refuses to Guess

This is an AI and data case study on a product I built and run myself. Pioneer Pennant Process is a college softball fit finder covering all 978 NCAA softball schools. It joins federal data on majors, cost, and outcomes to coaching contacts read from each school's own athletics website, and it has an AI that answers families' questions from that database. Building Pioneer Pennant Process was the same kind of work I do for small businesses as a fractional AI and data department, done on a system where I pay every bill and the support email comes to me.

The size is not the interesting part. The work went into the things the system refuses to do: store a coach it is not sure of, announce a coaching change nobody has checked, let its AI read tables it has no business reading, or show a family a number with two definitions. Those four refusals are the part of the build a business owner can copy.

There are no customer counts in this post, on purpose. What I can show is how the system is built and the calls I made building it.

Recruiting data is public, but it lives in 978 places

Most of what a softball family needs to pick a college is public, and almost none of it sits in one place. I learned that as a first-time recruiting parent, with my daughter starting the process and me knowing only what other parents had told me.

The U.S. Department of Education publishes cost, admissions, graduation rates, and earnings by major for U.S. colleges in the College Scorecard. The NCAA publishes which schools sponsor softball. The coaches, their email addresses, their camps, and their recruiting questionnaires sit on each school's own athletics website, laid out however that school's web vendor chose. Pioneer Pennant Process joins the three. It is part of Pioneer Pennant, the softball product line on my work page.

The figures are the ones the live stat band on pioneerpennant.com/process computes from the database, as of September 28, 2026. The app behind those figures runs more than 340 automated tests, and a crawler, a program that visits each school's website and reads the page, re-checks every head coach on a schedule. Most of the hard calls were about the crawler.

A wrong contact is worse than no contact, so the crawler leaves a blank before it guesses

The crawler's standing rule is that a wrong coach contact is worse than none, because a stored contact ends up in a high school athlete's email to a college coach. Athletics websites are hard for a machine to read, and they fail in ways that look plausible.

Site navigation reads like a name. Early versions of the crawler stored "Archived Stats" and "Sport Oversight" as head coaches, because each is two capitalized words sitting where a name usually sits. The crawler lifted three high schools out of coach biographies and filed them as people, and parsed a community college, one coach's former employer, as a person too. At one school the database carried the baseball coach as the softball coach, read off the wrong sport's roster page. During one change to the parser, the code that pulls a name out of a page, a school briefly got a head coach named "Media Guide." A regression test caught that one before it reached anything: 39 saved pages the new code has to read exactly the way the old code did.

Every one of those entries would have looked fine in a spreadsheet. None of them would have survived a coach reading the email.

I never loosened the rules to make the coverage look better. I made them smarter and left an honest blank where they still could not be sure. Small-school coaches wear several hats, and their title cell says so: in "Associate A.D./Head Coach" the person is the head coach, and in "Associate Head Coach" the person is not. Splitting each title into roles and judging each role alone gets both right. A day of parser fixes built around that idea took head coaches on file from 790 to 856. A person then filled in 26 more by hand. The other 96 schools are blank.

A blank is a decision, and the site says so out loud. The AI on the site is told to say "no coach on file" rather than suggest the job is open. Of the 96 blanks, 76 are Division III schools, mostly sites that build their staff pages with scripts a polite crawler cannot read. The email gap is the same story. When I checked the saved pages with a coach but no address in August, 76 of 132 did not publish an email address anywhere on the page. Finding those addresses is work for a person, or it cannot be done at all, and the parser fixes recovered none of them.

The camp links follow the same rule. A camp is where a college coach can actually watch an athlete play, so a link to each school's camps page is worth having. A first pass matched the word "camp" inside "Campus Life" and "Campus Map" and reported 83% coverage on a 30-school sample, when the honest figure was 67%. One school's "Clinics" link went to a cheerleading page. The detector is now built to refuse a doubtful link rather than accept it, and 695 schools carry one. Of those, 465 point at the athletic department's all-sports camp page, and the site labels them that way, because sending a family who expects softball to a basketball listing is worse than showing nothing.

Two sources I could have used, I left alone. The NCAA's own coach pages are the authoritative source, and they sit behind a bot-management service that turns away automated requests. Getting past that service would mean defeating a control the NCAA put up on purpose, so the crawler reads each school's own site instead. A camp registration platform would have been a clean source for dates and prices, and its terms forbid building a directory from it "whether by manual methods" or by bots. Those terms cover copying by hand as well, so the platform is not a source.

Nothing reaches a family until a person presses Publish

When the crawler notices that a school's head coach has changed, the change goes into a review queue, and nothing about it reaches a family until a person checks the school's page and presses Publish.

The crawler re-checks every head coach each Sunday night from June 1 through September 15, and on the first Sunday of the month the rest of the year. The weekly window closes just after September 1 of an athlete's junior year, the date NCAA Division I softball recruiting contact can start, which is when a dead coach address does the most damage. A full sweep takes about 30 minutes, at roughly 3.6 seconds a school.

Detecting a change was the easy part to automate. I was not willing to automate announcing one, because a scrape that becomes a public statement about a named person's job carries a different risk from a line in my own inbox. On the day the review step shipped, the database already held three detected changes that were false. The first weeks of a watch like this are the noisiest, because a school the parser read wrong once reads differently the next time, and the crawler records the difference as a change: parser corrections dressed up as coaching news.

So Publish is one button that does two things. It puts the change on a public page and emails the families who track that school. "Not a change" buries a false detection for good, and a change that has been mailed once is never mailed again, however many times it is republished. An email to a family cannot be unsent, which is the undo-cost test I use to decide where a person has to make the call.

The wording is a rule too, and an automated test enforces it. The public page says what the source lists: the school's staff page lists this person as head coach, where it previously listed that person. The page never says anyone was hired, fired, let go, resigned, or stepped down. Those are claims about events I did not see, and the neutral sentence stays true whether the school's website changed or the coach did.

The AI answers from the database, and the database enforces its limits

Ask, the question box on the site, answers from the database, and the limits on what it can read live in the database and the server instead of in a prompt the AI could argue its way around.

The first thing I learned about Ask was humbling. It felt canned to me, so I checked what was actually answering. The AI had never run in production. The server had no API key, the credential that lets a site call an AI model, and I confirmed that three separate ways. Every answer families had ever gotten came from the backup, a rules engine that could not compare two schools. Asked about the difference between Division II and Division III, it returned a list of twelve Division II schools. Adding the key did most of the fixing, and a rewritten prompt did the rest.

With the key in place, two guards are code. The AI writes its own database queries, and SQLite, the database underneath, checks every table and column a query touches while the statement is being prepared, before a single row is read. Three tables are allowed: schools, academics, and majors, and only named columns of each. The roster table, which holds 18,935 players by name, is not allowed, and neither are the staff table or the log of coaching changes. A clever query cannot route around that check, because the check does not read the query's text. The database itself refuses. Coach phone numbers are not on the list either, so a query that asks for them gets blanks back.

The second guard keeps whole categories of question away from the AI. Questions about recruiting contact rules, eligibility, transfer status, or an injury never reach the model. They get a fixed answer and a link to the site's guide, because those are the questions where a confident wrong answer costs a family the most.

A model answer takes 10 to 16 seconds and costs about four cents. Daily caps of 20 questions from any one internet address and 200 in total keep the worst possible day near $10. Each answer logs its token counts, the unit AI use is billed in, and never logs the question itself.

The site's usage analytics follow the same rule: limits in code, not promises in a policy. That matters more than usual on a product whose users are families of minors. The event log stores a timestamp, the kind of event, a page name from a fixed list, and an account number. The log never stores what someone typed or searched, an email address, an athlete's name, or an internet address. Deleting an account deletes its analytics too, through an explicit call, because the events live in a separate database file and SQLite cannot carry a delete from one database file into another. Without that one call, the right-to-delete promise in the privacy policy would have quietly become false.

Every number a customer sees has exactly one definition

Every figure on the site and in every customer email comes from one function that reads the database, because figures typed into copy drift, and they drift in public.

In August the welcome email told new families the database had contacts for 512 head coaches. The homepage, computed from the same database on the same page load, said 856. The email figure had been typed once and never updated. The emails now read the same function the homepage does, and when a figure is missing the sentence falls back to vaguer wording that is still true rather than a guess.

The next mistake was mine, and it is the more useful one. Checking the other printed figures that day, I re-derived them with the obvious database queries and got 968 schools with cost data, 22,100 salary figures, and 384 majors. The real definitions are stricter: cost counts only schools that actually report a net price, and majors and salaries count bachelor's degrees only. The true figures were 963, 21,480, and 371, and they had been right all along. I built a refreshed set of marketing cards on the three inflated figures before I caught the error the same day. An automated test now keeps the old figures out of customer copy, and the rule is short: for any number a customer sees, read the function, never a query you just wrote.

The same audit found a sentence that was worse than a wrong number. Printed material told families that athlete data "never leaves this computer," which stopped being true the day accounts existed. The line turned up in eleven places, not the three I first spotted, and it kept coming back because the product's own voice guide listed a version of it as a line to reuse word for word. Fixing the eleven copies without fixing the guide would have guaranteed a twelfth.

The softball analytics side of Pennant follows the same rule from the other direction. Its season report rebuilds 8,161 Division I games from free public data, and it never derives a count when an official one exists. Adding up box scores game by game reproduced only 4% of the top 25 home run totals exactly, running 10.3 home runs short on average, so every player total comes from the official leaderboard. Derived figures are reserved for what nobody publishes, like the count on every pitch.

What this AI and data case study does not show

This AI and data case study covers how the system is built and stops there. The gaps belong in the picture too: 96 schools have no head coach on file, the database covers NCAA programs only and no NAIA or junior college teams, the AI costs real money every time someone asks it something, and a coaching change reaches families only as fast as a person reviews it.

The same rules work inside a small business

The four rules behind Pioneer Pennant Process carry straight into a small business, because none of them are about softball. They are about what an automated system is allowed to do with other people's names and money.

• Leave a blank before you guess. That is the rule I would put on a customer list before any AI writes to the people on it.
• A Publish button belongs on anything that speaks for a business in public, like a review reply or a price change.
• A short list of allowed tables is how an AI reads the data it needs and never touches payroll or customer notes.
• One definition per number is why the dashboard and the invoice should never disagree.

Pioneer Pennant Process is where I practice those rules on my own time and my own money. For a small business, the same work happens inside a monthly contract, and how that works fits on one page.