Solwey

As AI moves from experimental "cool feature" to core enterprise infrastructure, one question is dominating C-suite conversations: How do we know we can trust it?

The answer lies in Evals (Evaluations). Think of them as the modern version of Test-Driven Development (TDD) for the age of LLMs. Because models are nondeterministic - meaning the same prompt can yield different results every time - traditional software testing isn't enough. Evals provide the structure needed to turn that unpredictability into a measurable, scalable system.

Designing an effective evaluation isn't just a technical task; it's "human problem engineering." Before writing code, you have to define what "good" looks like for your specific use case. Is it accuracy? Safety? Tone? Compliance?

In practice, this looks like a layered approach. Online Evals act as real-time guardrails, checking AI responses for compliance before they ever reach a customer. Offline Evals review performance trends across thousands of conversations to identify "context rot" or subtle drifts in behavior over time.

A common mistake is confusing Benchmarks with Evals. Benchmarks tell you how a model performs "out of the box" compared to its peers. Evals tell you how your product performs using your specific prompts, your data, and your unique business logic.

For regulated industries like finance and insurance, the margin for error is zero. Moving from a prototype to a production-ready agent requires a data-driven feedback loop. By running A/B tests on prompt versions and tracking metrics over time, teams can iterate with the same discipline used in traditional software engineering.

The "move fast and break things" era of AI is ending. Success now belongs to the organizations that can prove their AI is safe, reliable, and compliant through rigorous, continuous evaluation.

The goal is to ensure your technology supports your growth rather than defining its limits.

Read more in our Blog: https://www.solwey.com/posts/the-role-of-evals-in-better-ai

The American manufacturing sector is facing a projected gap of 2.1 million unfilled jobs by 2030. While headlines often focus on automation or offshore competition, the real crisis is a human one: a widening "perception gap" and a need for a culture shift.

For decades, manufacturing was seen as a last resort - repetitive, rigid, and manual. But modern industry is fueled by innovation, high-tech problem-solving, and Industry 5.0 - a model where technology serves to elevate human potential rather than replace it.

While tax breaks and higher wages make great headlines, they don’t solve for the human element. Workers today stay where they feel valued. According to self-determination theory, retention is driven by three key needs:

-Autonomy: Giving employees control over their decision-making.
-Relatedness: Fostering a genuine sense of community and belonging.
-Competence: Investing in upskilling so workers can master their craft.

The roadmap for leadership to close the talent gap, manufacturers need to rethink their approach to human capital:

-Rethink Job Structures: Move away from rigid descriptions toward adaptive models that encourage initiative.
-Broaden the Talent Pool: Strengthen connections with vocational schools, create dedicated pipelines for veterans, and actively mentor underrepresented groups like women in leadership.
-Technology as a Partner: Use AI, data analytics, and robotics not to limit humans, but to handle the "dull, dirty, and dangerous," allowing workers to focus on creative strategy.

The industry is at a turning point. Manufacturers that invest in their people through learning and empowerment will be the ones that define the next chapter of industrial success.
The goal is to build a workplace where people don't just work - they thrive.

Read more in our Blog: https://www.solwey.com/posts/whats-driving-the-manufacturing-talent-shortage-and-the-path-forward

Why plugging AI into broken data makes things worse

AI doesn’t fix data problems.
It amplifies them.

When data is fragmented, inconsistent, or poorly understood, adding AI doesn’t create clarity - it creates faster, more confident confusion. The outputs look polished, but they’re built on shaky foundations.

This is where many AI initiatives go wrong. Teams rush to deploy models before aligning on definitions, ownership, and data flow. AI then produces insights that don’t match reality, eroding trust and slowing decisions instead of improving them.

The irony is that the better the AI, the worse the outcome can be. High-quality models generate convincing answers even when the inputs are flawed - making errors harder to spot and easier to act on.

The companies that succeed don’t start with AI.

They start by fixing data flow, agreeing on what “truth” means, and designing workflows around decisions. Only then does AI become a force multiplier.
AI on broken data doesn’t just fail to help.

It makes the system louder, faster, and more wrong.

The phrase "custom code" usually brings to mind a dark room and endless lines of green text. In reality, it’s a business decision about ownership and limits.

Most companies start with great off-the-shelf tools like Shopify, Airtable, or Zapier. These are perfect for validation. But eventually, you hit a wall where you’re "babysitting" your tools instead of growing your business.

If your automation requires constant manual intervention, or your integrations are failing under load, it’s a sign that your technology is holding you back.

The Secret of Great Engineering Great software isn't built 100% from scratch anymore. It’s about balance. We use existing APIs for things like payments or SMS so we can focus the "custom" energy on the features that actually define your unique value.

3 Mistakes to Avoid When Going Custom:

1. The "One-Man Army" Trap: Relying on a single freelancer is a massive risk. If they go MIA, your project dies. You need a team for continuity and breadth of expertise.
2. Overbuilding Too Soon: You don’t need a system that supports millions of users on day one. Build for the load you have, but plan the architecture for the growth you want.
3. Seeking Perfection: Start with an MVP. Building incrementally is cheaper, faster, and allows you to pivot based on real user feedback.

The goal is to ensure your technology supports your growth rather than defining its limits.

More in our Blog: https://www.solwey.com/posts/when-custom-software-solves-real-business-challenges

Thinking about applying to a startup accelerator?
It’s a massive commitment, and the "investor intros" are only half the story.

Most founders see accelerators as a shortcut to funding, but the real ROI is often the forced evolution of your business.

These programs condense years of networking and trial-and-error into a few intense months. If you’re a first-time founder, that hands-on education is gold - it helps you navigate the legal and financial pitfalls that usually kill young companies.

But here’s the reality check: The "cost" isn't just the equity.
The biggest risk is time. Every hour you spend in a workshop is an hour you aren’t building your product or talking to customers. If the program isn't a perfect fit for your industry, it can quickly become a distraction. You have to be ruthless about protecting your focus.

How do you actually get in?

Accelerators aren't looking for "ideas" - they’re looking for ex*****on. To stand out, you need a tangible Minimum Viable Product (MVP) and a team that’s clearly all-in. They want to see that you’ve already started the journey and just need their fuel to go faster.

More in our blog: https://www.solwey.com/posts/the-real-value-and-risks-of-joining-a-startup-accelerator