I have a buddy with a software business. He has 12 years or so of Github issues,...

I have a buddy with a software business. He has 12 years or so of Github issues, emails and support requests over that time, along with versions and licenses. So he does all his diligent work, and scrapes all his content into JSON files, and then feeds it all to ChatGPT so it can learn his business. Magically (expected) ChatGPT imposes order on the content, and soon he has a workable chat tailored to his business. So he can now unleash this useful AI support tool to his customers. Except after about a week, the whole thing fell apart and started generating nonsense! The model stopped making sense. He is paying ChatGPT their fees for using their service, and he is not impressed that this is what happens. Why is it that a service like ChatGPT, supposedly a leader in consumer AI, cannot seemingly service their customers? Is this a common problem? I have seen quite a few memes about how these services all start out smart but then get dumb pretty quick, but I mostly ignore memes. Except when they come true! Anyone have any obvious things to look for in figuring out how to get ChatGPT back on track and working well for the long haul? I was gonna suggest he just use ollama and serve up his data with a fixed model and not use a service, but that seems like a lot of work to do again.