Build a proof of concept
Last updated
Last updated
You have a solution in mind, it’s time to start testing feasibility and support your engineering team. Here’s how you can bring in the required product context into the building process.
Identify the sources for the LLM to pull from.
Some questions to get you started—
What type of queries would you anticipate from users?
Do you have a knowledge base that will act as a source? Any other form of documentation?
Does it need additional interaction level information about the user’s behaviour?
Is there going to be a collaborative back-and-forth between the user and the LLM? If so, how should you retain context in an LLM<>user collaboration session?
Articulate what a meaningful response looks like for your use case. Identify what kind of fine-tuning or prompting you’d need to get to this response reliably.
Bring in stakeholders who have context of the product to collaborate on the prompts and examples.
Do this if a prompt is not effective. It will improve accuracy, give a deeper understanding of the product space, and be able to respond to a wide input variety.
Create a dataset of ideal responses. Creating good training data is a design and copy responsibility. Generating high quality example data is critical to build the best experiences. Learn more about crafting responses
Keep them specific and context rich. Prompt engineering uses natural language so it’s quite accessible, and is core to the product experience. If you’re using OpenAI, see how far you can get with just testing out your idea on the before you actually start pulling from all your sources.
From