Fastdatascience.ai

What are open source LLMs and should you deploy one?

Open source LLM's are language models that are available to download and use without requiring a subscription or fee for use...

These models work similarly to the foundational models that underlie ChatGPT and Google's Gemini (formerly Bard) in that they can produce text content in response to natural language direction...

That said, these open source models often do require further fine tuning in order to meet the needs of more specific applications...

And "open source" doesn't mean that these models have no cost associated...

As with any large language model, compute resources required to run them are not negligible...

For example, the new Smaug-72B-v0.1 model released by Abacus.ai has a minimum requirement of running on the m5 series EC2 instances, ml.g5.2xlarge more specifically, which comes with a cost of about $1.21 per hour to stand up on AWS...

But if data security is a concern and you have the ability to fine tune for your use case, then open source models can offer a valuable option for integrating AI for secure enterprise use cases...

Meet ‘Smaug-72B’: The new king of open-source AI Abacus AI has released "Smaug-72B," a new open-source AI model that outperforms GPT-3.5 and Mistral Medium on the Hugging Face Open LLM leaderboard.

Getting value for the everyday with generative AI is easy, almost too easy…

And its ease is causing more and more employees to turn to these tools, even without their company’s approval, in the shadows behind the firewall…and has fittingly become known as shadow AI…

Shadow AI, the unauthorized use of AI tools within your organization. While tempting for its quick fixes, its hidden dangers can be catastrophic…

Let's shed some light:

• Data breaches & security risks: Unsanctioned tools often lack robust security, increasing vulnerability to leaks and cyberattacks. Imagine confidential customer data exposed due to a "shadow" marketing project…

• Bias & discrimination: Unvetted AI can perpetuate harmful biases present in datasets or algorithms, leading to unfair treatment of employees or customers. Think discriminatory hiring practices fueled by a rogue recruitment tool…

• Compliance nightmares: Shadow AI can violate data privacy regulations, exposing your business to hefty fines and reputational damage. Picture GDPR violations stemming from an unregulated sentiment analysis tool…

So, how can businesses address this growing concern…

1. Start with transparency and education. Although even the simple user interfaces that accompany tools like ChatGPT and Google’s Bard can be powerful in their simplicity, using them opens risk to sharing confidential data. Educating employees on these risks and establishing clear policies for proper use are essential for ensuring employees can still leverage the tools…

2. Build clear and diverse governance practices. As employees are educated on the risks and benefits, codify what is learned by publishing governance practices that help to delegate responsibility across the organization…

3. Build in technical safeguards. Dealing with the ease of access makes setting up technical safeguards the most challenging. And different companies will have different requirements for how important it is to set up tighter or looser technical walls. For the truly risk averse, it may be useful to build and train models that score prompts for risk before allowing them to be passed to generative AI for content…

Remember, shadow AI isn't just an IT issue; it's a strategic one…

Shadow AI in the 'dark corners' of work is becoming a big problem for companies Employees who aim to innovate with their own unapproved artifical intelligence tools often do so with unintended consequences.

What if ChatGPT went away...for good?

That is what a new lawsuit by the New York Times against OpenAI for copyright infringement could entail...with emphasis on the "could"...

The issue being addressed is that the NYT claims to have evidence that ChatGPT was not only trained on copywritten content but that it also has generated copywritten content written by writers at the NYT...

And while losing ChatGPT would be a significant setback for the hundreds, or even thousands of businesses leveraging the technology, there are some crucial lessons to learn regardless of how this case unfolds...

In short, businesses need to be extremely careful in curating both the use cases and content being generated for those use cases when deploying generative AI...

Moreover, a careful consideration of possible consequences must be thought through before we lean to hard on a solution with generative AI built-in...

Perhaps businesses should leverage architectures that not only fact check outputs but also check for plagiarism to be used as a way of scoring the originality of the generated work...

In this way, generative AI can be thought of like a student who may not always know when information it has learned isn't being paraphrased enough to warrant originality...

Ultimately, and as the article points out, it is unlikely the government will request a full deletion of ChatGPT's trained models but I would expect to see more regulation be considered as governments get up to speed with the social impact of generative AI...

What the New York Times lawsuit could mean for ChatGPT Can a federal court actually order the destruction of ChatGPT? Yes. Will it? Maybe.