What I Learned Pushing Prompt Engineering to the Limit
<p>I spent the past two months building a large-language-model (LLM) powered application. It was an exciting, intellectually stimulating, and at times frustrating experience. My entire conception of prompt engineering — and of what is possible with LLMs — changed over the course of the project.</p>
<p>I’d love to share with you some of my biggest takeaways with the goal of shedding light on some of the often unspoken aspects of prompt engineering. I hope that after reading about my trials and tribulations, you will be able to make more informed prompt engineering decisions. If you’d already dabbled in prompt engineering, I hope that this helps you push forward in your own journey!</p>
<p>For context, here is the TL;DR on the project we’ll be learning from:</p>
<ul>
<li>My team and I built VoxelGPT, an application that combines LLMs with the FiftyOne computer vision query language to enable searching through image and video datasets via natural language. VoxelGPT also answers questions about FiftyOne itself.</li>
<li>VoxelGPT is open source (so is FiftyOne!). All of the code is available on GitHub.</li>
<li>You can try VoxelGPT for free at gpt.fiftyone.ai.</li>
<li>If you’re curious how we built VoxelGPT, you can read more about it on TDS here.</li>
</ul>
<p>Now, I’ve split the prompt engineering lessons into four categories:</p>
<ol>
<li>General Lessons</li>
<li>Prompting Techniques</li>
<li>Examples</li>
<li>Tooling</li>
</ol>
<h1>General Lessons</h1>
<h2>Science? Engineering? Black Magic?</h2>
<p>Prompt engineering is as much experimentation as it is engineering. There are an infinite number of ways to write a prompt, from the specific wording of your question, to the content and formatting of the context you feed in. It can be overwhelming. I found it easiest to start simple and build up an intuition — and then test out hypotheses.</p>
<p><a href="https://towardsdatascience.com/what-i-learned-pushing-prompt-engineering-to-the-limit-c40f0740641f">Read More</a></p>