An in-depth look at the foundation models and tools used to develop the US Open fan experience

For more than three decades, teams of developers and data scientists from IBM Consulting® have collaborated with the United States Tennis Association (USTA) to provide an engaging digital experience for US Open tennis fans.

Let's take a deep dive into this year's innovations across two generative AI projects that leverage IBM's versatile family of enterprise-ready Granite™ foundation models, among other models. We'll also look at how the team used IBM watsonx Code Assistant™ to accelerate code generation and improve productivity and collaboration.

We spoke to project lead Aaron Baughman, IBM Fellow, Master Inventor and IBM Quantum™ Ambassador to learn how IBM is driving insights at the US Open.

Project 1: The content engine

Providing up-to-date story coverage for the hundreds of matches across the men's and women's singles competition is a massive undertaking. But by leveraging the hybrid cloud architecture with Red Hat® OpenShift® that IBM Consulting has built over years of working with the USTA, the development team can quickly create, test and implement new automated workflows to address such challenges. This year's content engine is one such workflow.

The content engine produces three main outputs: bullet-point descriptive texts before and after every singles match, spoken commentary and subtitles for match highlights, and multi-paragraph Match Reports that provide descriptive summaries and analysis about completed matches.

The underlying data

The system draws from myriad data points including world rankings going into the tournament, and ongoing match play: 128 matches in the first round, 64 in the second, and so on, down to the final two men's and women's Championship matches. The Likelihood to Win predictions for each singles match-a popular stat for discussion among fans and media commentators alike-are also generated, using AI analysis of recent performance.

The generative AI system

After that process is complete, the generative AI system creates pre-match bullet points. "Early each morning we process the day's scheduled matches through our agentic architecture," says Baughman. "The Granite 13b chat model produces our bullet points; we use a few-shot technique where we give it examples to follow and deliver similar output." The pre-match bullets give insights based on rankings, head-to-head results and player biographies, providing fans with context for the match ahead, and are published on the website and app.

When a match finishes, the system generates text descriptions of what happened-drawn from stats such as aces, break points won, double faults, winners and shot speed. These descriptions are then transformed into natural language bullet points by generative AI models, including IBM Granite, which are hosted on the IBM® watsonx™ AI and data platform.

Next, the Match Reports are created. These reports draw on trusted US Open data and use the combined power of Granite and other models hosted on IBM® watsonx.ai™ to create long-form summaries. "The Match Reports summarize who played, what happened, and help explain why a player won," says Baughman. The narratives are then reviewed, edited and published on the app and website and by the USTA editorial team.

The benefit

Before the content engine existed, editors had to spend hours watching replays and interpreting scorelines and stats before they could begin writing longer stories to publish in US Open media channels. Match Reports provide at-a-glance story arcs and highlights so they can begin writing right away. And for the first time ever, the USTA editorial team will be able to publish a match report for every men's and women's singles match this year.

Increased development speed and improved collaboration with watsonx Code Assistant

IBM® watsonx™ Code Assistant™ provides enterprise-grade code generation, providing snippets and functions to speed application modernization, automation and scaling. Trained on Granite foundation models, the assistant provides AI-generated recommendations based on existing source code and responds to natural language requests.

Watsonx Code Assistant helped accelerate development of substantial parts of the content engine. Using a code plug-in within their integrated development environment, developers could chat with the assistant through a sidebar panel. They were able to ask it questions such as how to randomly select text from an array, then get a recommended code snippet, which could be copied and customized to the data they were using.

In addition to code generation, watsonx Code Assistant provides valuable insights to teams working together, trying to understand and build on existing code.

"For example, if somebody wrote a function or a method, I didn't have to read through all the code to figure out what it did," says Baughman. "I could highlight the block, and the assistant would summarize what the code does. It also helped us create comment blocks that described at a high level what each variable represents. We could more easily assess what the methods would return and how to use them."

If the output is still unclear, the assistant also responds to follow-up questions. The application also allows for immediate thumbs-up or thumbs-down feedback on the output, improving the performance of the tool.

As the collaboration and conversation with watsonx Code Assistant progresses, users can either build on the existing chat history or reset it for new contexts and questions.

"Say you're writing a fact-checker for generative AI, some of the output you like, some of it you don't," says Baughman. "You are in control, choosing the best code that makes sense for you. If there's a code snippet you don't understand, you can ask a question about it. It's almost like a choose your own adventure for development."

Project 2: Audio commentary

Introduced last year, AI-generated audio commentary provides automated voiceovers and subtitles for every singles match highlight reel shown on the US Open website and app. This feature uses a combination of models, including Granite 13b chat models, to create complex tennis language in support of generated commentary.

Enhancing personality and color of synthetic speech

This year, a key goal was to make the audio commentary more natural and human. The team experimented with two variables: top k, a parameter that controls the number of possible answers the model should consider, and temperature sampling, used to adjust the probability distribution of possible answers. These levers help ensure that the model generates a more human variety of phrases rather than the most probable and repetitive ones.

The other way the team could affect personality was through the prompts, which are system-level instructions delivered in natural language that tell the model what to do. "We instructed the model, 'Create interesting commentary about this match,'" says Baughman.

Testing and human review

In the testing phase, the teams reviewed and fine-tuned the commentary. "It's a balance between artfulness and control-it's not an easy decision," says Baughman. "The more control, the more development effort. The less control, the less development effort but the riskier it is. The use case informs where you land."

The next step is going from text to speech, where it is essential to make the voices sound convincingly human. Through experimentation and many different runs, the teams made sure the voices are clear and have the right prosody-that is, the pitch and speed match the nature of what is being said.

Inference and output

When these balances are found, the inference and output processes happen largely unsupervised, in near real time. Five events trigger commentary creation: match end, match start, set point end, game point end and match point end-about 9,000 commentary events across matches in total. When an event happens, the commentary systems get a message and execute generation of commentator phrases. The sound files are then integrated into video highlight reels.

Experience that informs innovation

By working closely with the USTA over the years, the IBM Consulting team explores new ways to create magnetic fan experiences and improve the productivity of the US Open digital team. It's also an opportunity to showcase powerful new tools, such as watsonx Code Assistant and the family of Granite foundation models.