We at DataSlush are working on multiple proof-of-concept projects on large language models. It was interesting to learn about the application of LLMs in various business domains. In this blog post, I am going to share some of the learnings that we had while utilizing LLMs for different problems
1. Be reasonable on what to expect from LLMs
LLMs are not 100 % deterministic, so you need to be very much careful while using them. It is good at summarizing the knowledge it has, but would not work on intelligent calculations. So ensure that you have the right use case where you expect LLMs to generate some content based on instructions that don’t involve too much of math calculations. We faced this issue while working on POC of Math Tutor.
2. Larger problem should be divided into smaller problems:
What we have seen so far is that if we ask LLM to generate output for a large problem comprising of multiple small problems, It starts hallucinating more. So the solution to it that we believe has got us good results is to divide large problems into smaller ones. If you create multiple prompts and combine the output of it to solve a larger problem, it will work well than a single prompt.
3. Think for corner/edge cases first even before solving the main problem
In many applications, we have observed that we have ended up working more on edge/corner cases than the main problem. You will require to think wisely about all the edge cases first as they will become headache to handle at a later stage of development. We recommend to thoroughly defining the list of edge cases
4. Apply processing layer for both input and output
If there is no processing layer implemented for both input and output. LLMs will start creating unwanted output. In a few applications we have seen that users input is required to be parsed in a specific way before sending to LLM. The same is also applicable to the output generated by LLM
5. Give detailed instructions in the prompt
We have seen that providing detailed instructions on a specific task has helped to generate better results from LLM. We have religiously followed the GPT best practices by OpenAI and it has helped us immensely in obtaining structured output
6. Utilise tools like Langchain
Langchain has played a very much vital role in our proof-of-concept projects. It has helped us to make our iterations faster in development. We highly recommend using langchain for LLM applications development
In my next post, I will be covering some of the points w.r.t. productionising the LLM application. Stay tuned till that time!
