Most language models are built using a combination of text-data collected off the internet and those found in legal text (either publicly or privately) which serves as a foundation for the models training. GPT-4’s neural network was famously trained using 1 petabye (1 million gigabytes ) of text from the internet and has 1.8 trillion parameters. Even with all that, GPT is prone to hallucinations and model degradation.
It’s probably safe to say that there’s nowhere near this amount of publicly available legal data which means there’s no way of creating as accurate of a model from the ground up. Instead, most companies in the space opt to use GPT or other language models as a base and layer in another fine-tuned model that utilizes proprietary legal datasets.