Insightful stories that revolve around technology, culture, and design

From assisting with surgeries and pharmaceutical development to improving patient communication and disease tracking, Artificial Intelligence is shaping how we address public health’s toughest challenges all while transforming care delivery on a global scale.

During critical moments like the COVID-19 pandemic, AI demonstrated its ability to forecast outbreaks, optimize vaccine distribution, and combat misinformation. 

Beyond crisis scenarios, AI continues to revolutionize public health by analyzing vast datasets, identifying disease risks early, and guiding resource allocation to maximize impact.

However, with great potential comes responsibility. Public health systems must navigate through significant challenges to fully realize the benefits of AI. 

This blog explores how AI is advancing personalized care, addressing systemic challenges, and paving the way for smarter, more equitable public health solutions.

How it all started

The journey of AI in public health spans over five decades, marked by significant technological leaps.

It began in the 1970s with systems like MYCIN, which used rule-based algorithms to help doctors with diagnoses and treatments. While innovative, these early systems were limited in handling large data or adapting to new situations.

By the 1990s, advances in computing enabled AI to process larger datasets, paving the way for systems like the Early Warning and Response System to detect disease outbreaks.

 In the 2000s, AI used machine learning for public health trends, with solutions like Google Flu Trends estimating flu activity via search data. 

In 2019, BlueDot demonstrated AI’s power by detecting COVID-19 nine days before official warnings.

Today, AI maps disease hotspots, optimizes vaccine distribution, and predicts risks, supported by organizations like the WHO and CDC. 

Despite progress, challenges like data privacy and equitable access remain, highlighting the need for careful integration to realize AI's transformative potential in public health.

The hurdles we face

Privacy concerns

Health data is highly sensitive, and its use in AI raises substantial privacy risks.

Unauthorized access or breaches can lead to misuse of personal information, as seen in debates over tech companies accessing NHS archives for AI research.
‍

Access disparities

‍
AI’s benefits often favor regions with advanced infrastructure, leaving low-income areas behind. 

Many communities lack the technology and connectivity to leverage AI tools, further widening global health disparities. 

Algorithmic bias

‍
AI models are only as good as the data they are trained on. If that data lacks diversity, the results can perpetuate biases, leading to unequal healthcare outcomes. 

For example, models trained predominantly on Western datasets may fail to perform accurately for other populations.

Ethics and regulation

‍
Clear guidelines for the ethical use of AI in healthcare are still in development. 

The FDA’s efforts to create tailored regulations for AI-based medical devices highlight the need for oversight.

Transparency in AI models

‍
Many AI systems operate as “black boxes,” with decision-making processes that are hard to interpret. This lack of transparency can lead to mistrust among healthcare providers and patients. 

Developing interpretable AI systems is crucial for their broader acceptance 

Interoperability

‍
AI solutions often struggle to integrate with existing healthcare systems, particularly in regions with diverse infrastructure. 

Creating solutions that are compatible across systems is critical for effective implementation.

Cost and scalability

‍
AI technologies can be expensive to develop and implement, particularly in resource-constrained settings. 

Finding cost-effective ways to scale AI solutions remains a major hurdle 

Cultural and linguistic barriers

‍
AI tools must account for cultural and linguistic differences to be effective globally. 

Without proper adaptation, models trained on one region’s data may not perform well elsewhere .

Workforce readiness

‍
Healthcare professionals often lack training in AI technologies, limiting their ability to use these solutions effectively. 

Including AI education in medical training programs is crucial for successful adoption.

Long-term data reliability

‍
AI relies on historical data, which must be continuously updated to remain accurate. 

How AI is transforming public health solutions

AI offers transformative solutions in public health, addressing the issues in early detection, personalized care, resource management, and health communication. 

Here are the key areas where AI is making a difference

Early disease detection and surveillance
‍

AI Platforms like ProMED  analyze large datasets like medical records and environmental data to detect outbreaks early. 

Predictive models assess disease spread patterns, aiding in timely interventions. The  World Health Organization’s Epidemic Intelligence from Open Sources (EIOS) supports global disease surveillance using AI.

Personalized healthcare
‍

AI tailors treatments by analyzing patient-specific data. For instance, mental health apps like Woebot use AI to offer real-time support for anxiety and depression.

AI in cancer care helps doctors design personalized treatment strategies. Research, such as a study published in Nature Medicine, shows AI’s ability to detect breast cancer more accurately than radiologists.

Resource optimization
‍

Predictive models improve resource allocation during emergencies. AI frameworks like DeepMind’s Streams App optimize hospital bed usage and patient prioritization.

For example, Operation Warp Speed utilized AI for supply chain optimization

Improving access to reliable information
‍

AI chatbots, like Ada Health, provide accurate health guidance to patients and caregivers, reducing misinformation.

Post-discharge monitoring digital healthcare platform, such as Babylon Health, helps track patient recovery and ensure continuity of care.

Predictive analytics for long-term planning
‍

AI systems analyze environmental and social data to predict long-term health trends. 

For example, IHME’s Global Burden of Disease project leverages AI to forecast health challenges like chronic diseases.

Predictive analytics also support planning for emerging threats like antimicrobial resistance (CDC on AMR).

Supporting health equity
‍

AI addresses health disparities by identifying underserved communities. 

Projects like Data for Good at Meta use AI to map healthcare access and needs in vulnerable populations.

Vaccine allocation models, such as those developed by UNICEF, use AI to prioritize high-risk groups.

Medical research and drug development
‍

AI accelerates drug discovery, reducing time and costs. Platforms like Insilico Medicine use AI to predict viable drug candidates.

Machine learning models, such as those developed by IBM Watson Health, assist in optimizing clinical trials and participant selection.

By leveraging these AI solutions, public health systems can enhance efficiency, equity, and outcomes on a global scale.

Looking forward

The stakes are enormous, but so is the opportunity with AI to transform how care is delivered to the most vulnerable population.

With the right focus on equity, collaboration, and responsible use, AI can bridge gaps in healthcare access, empower decision-makers, and make public health systems smarter.

For instance, AI algorithms trained on vast datasets can outperform experienced doctors in detecting conditions like skin cancer and breast cancer risk. 

AI models like Sybil are leading the way in predicting health outcomes using current and future data trends, paving the path for personalized, forward-looking care.

Studies at institutions like MIT show that hybrid human-AI models excel in scenarios such as identifying cardiomegaly in chest X-rays.

Beyond diagnostics, AI applications are making global health systems more efficient, from allocating resources to forecasting disease outbreaks. 

However, the road ahead requires a strong focus on equity and community-driven solutions. AI should benefit everyone, not just those with privileged access. 

By building partnerships and adhering to principles of fairness and accessibility, AI can help create a healthier, more equitable world.

‍

AI agents are changing how systems operate. They don’t just follow instructions; they make choices, adapt to situations, and get better with every interaction. For companies building modern, connected workflows, that shift moves automation from running processes to managing outcomes.

Most businesses already use bits of automation in isolation. Chatbots answer questions, dashboards track metrics, and scripts move data from one place to another. AI agents bring these parts together. They combine logic, context, and learning so systems can coordinate, reason, and act with purpose.

This blog walks through the main types of AI agents, how they work behind the scenes, and where they are already shaping results. It offers a grounded view of how intelligent systems make work faster, more connected, and more deliberate.

‍

‍

‍

AI Agents vs AI Assistants vs Bots

Many systems today can chat, automate, or take action, but not all are AI agents.

• Bots handle simple, rule-based tasks like answering FAQs.
• Assistants perform guided actions, such as scheduling meetings or setting reminders.
• Agents work with goals. They plan, decide, and act independently, using data and reasoning to get results.

This difference in autonomy and decision-making is what makes agents valuable in business settings.

How AI Agents work

Behind every agent is a simple process: sense, think, act, and improve.

• Brain: Understands context and intent.
• Planner: Breaks a goal into smaller, actionable steps.
• Memory: Keeps past information to make better choices next time.
• Tool Use: Connects with APIs or apps to perform real tasks.
• Reflection: Learns from outcomes to improve over time.

These pieces together form what’s called an agentic system, software that can act with purpose instead of only reacting.

Simple reflex agents

Simple reflex agents use predefined rules to make decisions without considering historical data or future outcomes.

Their functionality is rooted in condition-action rules such as, “If this, then that.” 

While these agents are effective in predictable scenarios, they cannot adapt to unforeseen changes or manage complex tasks.

Example: A motion-sensing light is an example of a simple reflex agent. 

It uses a sensor to detect movement within its range and turns the light on when motion is detected. After a set period of no further movement, it automatically turns the light off. 

The light works on the rule: 'If motion is detected, turn on; if not, turn off.' However, it cannot handle complex scenarios like distinguishing between a person and a moving object like a pet.

Strengths: Simple reflex agents operate reliably in predictable environments with clear rules, like turning on lights when motion is detected or maintaining a set temperature.

These agents are quick and efficient because they follow pre-set rules without needing complex reasoning.

Limitations: However, they struggle in more complicated or unpredictable environments. 

They cannot adapt to change or handle situations where information is incomplete or unclear. 

For instance, a thermostat won’t know what to do if the heating system breaks down, and a motion-sensing light can’t differentiate between a person and a moving curtain. This lack of flexibility and awareness makes them unsuitable for tasks that require learning or decision-making in changing conditions.

Model-based reflex agents

Model-based reflex agents are like smart problem-solvers that use a mental map of the world to make better decisions. 

These agents use an internal map to keep track of their surroundings and determine their next actions 

They work well in changing environments but still depend on rules set in advance. 

They’re not fully independent thinkers, they follow their programming but do so in a smarter, more informed way.

Example: Robotic vacuum cleaners demonstrate model-based reflex agents, using sensors to map rooms and avoid obstacles.

 It uses its "mental map" to handle changes, like moving a chair, and still does the job.

Advantages: Model-based reflex agents are capable of working in environments where they don’t have complete information. 

They use an internal model of the environment to fill in the gaps, helping them make decisions even when they can’t see or access all the data. 

This ability makes them effective in partially observable environments. 

Additionally, they can adapt to changes by updating their internal model based on new observations, allowing them to respond to dynamic situations.

Challenges: The performance of model-based reflex agents heavily depends on the quality of their internal model.

 If the model is incomplete, outdated, or inaccurate, the agent might make poor decisions or fail to perform its task. 

For example, if the internal model doesn’t account for unexpected events or changes in the environment, the agent might not respond effectively. 

Maintaining an accurate and comprehensive internal model requires careful design and regular updates, which can be complex and resource-intensive.

Goal-based agents

Goal-based agents are intelligent systems designed to achieve specific objectives. 

Goal-based agents plan step by step, analyzing options and predicting outcomes to achieve specific objectives.

Example: A smart sprinkler system is a good example of a goal-based agent. 

Its goal is to water the garden efficiently while avoiding waste.  It considers factors like weather forecasts, soil moisture levels, and the time of day to decide when and how much water to use.

If it detects unexpected rain or a change in weather conditions, it adjusts its schedule to ensure the garden is watered appropriately without wasting water.

By constantly evaluating its actions against its goal, it ensures optimal performance.

Benefits: Goal-based agents are excellent at solving problems because they focus on achieving specific outcomes. 

They handle complex tasks effectively by thinking ahead and adapting to changing conditions

They are also good at adapting to new challenges, as they can adjust their actions when the environment changes or when unexpected situations arise. This flexibility allows them to remain effective even in dynamic or unpredictable conditions.

Applications: Goal-based agents are widely used in tools and systems that require planning and decision-making. 

Navigation systems, like GPS, use them to calculate the best routes by considering factors like distance and traffic. 

Scheduling tools rely on these agents to organize tasks efficiently, ensuring deadlines are met and resources are used effectively. 

Strategic planning software in businesses and industries uses goal-based agents to create detailed action plans and make data-driven decisions, optimizing processes for better outcomes. 

These applications showcase how goal-based agents simplify complex tasks and improve efficiency across various fields.

Utility-based agents

Utility-based agents use a utility function to select actions offering the greatest benefit, factoring in efficiency, cost, and time.

These agents don’t just focus on achieving a goal; they aim to do it in the best possible way. 

They compare different options, calculate their utility, and pick the action that gives the highest value. 

Example: Ride-sharing platforms are a great example of utility-based agents in action. 

When a rider requests a trip, the system evaluates multiple factors to find the best match with a driver. 

These factors include the driver’s proximity to the rider, the estimated wait time, the cost of the trip, and current traffic conditions.

The platform calculates a "utility score" for each potential match, which represents how beneficial or efficient that pairing would be. 

For instance, a nearby driver with minimal traffic and a shorter wait time might be assigned a higher utility score. Once the system compares all possible matches, it selects the one with the highest score, aiming to balance speed, cost, and overall efficiency for both the rider and the driver. 

This approach ensures a well-optimized experience while managing the needs of both parties effectively.

Advantages: Utility-based agents excel in handling situations where multiple priorities or factors need to be balanced. 

They evaluate all possible actions and assign a "utility" value to each option, which measures how beneficial or effective that action is. 

By choosing the action with the highest utility, they ensure the best possible outcome. 

This makes them especially effective in complex scenarios where trade-offs between efficiency, cost, and time need to be considered.

Applications: These agents are widely used in fields that require careful planning and optimization.

Utility-based agents optimize logistics, financial planning, and resource allocation by evaluating costs, delivery times, and risks. 

For resource optimization, utility-based agents allocate limited resources, such as energy or manpower, in the most efficient way possible, ensuring that goals are met with minimal waste. 

This ability to make calculated decisions based on multiple variables makes them indispensable in these domains.

Learning agents

Learning agents are advanced AI systems that can improve themselves over time by learning from their experiences. 

Learning agents adapt and improve as they gather information. These agents analyze past actions and their outcomes to understand what works and what doesn’t. 

This learning process allows them to make smarter decisions and perform tasks more effectively as they gain experience.

 These agents consist of four components:

• Learning element: Acquires new knowledge from the environment.
• Critic: Evaluates the agent’s actions and provides feedback.
• Performance element: Determines actions based on learning.
• Problem generator: Proposes exploratory actions to improve future performance.

Example: Spotify's music recommendation system is a great example of a learning agent. 

It learns your music preferences by analyzing the songs you listen to, how often you play them, and whether you like, skip, or save them. Over time, it uses this information to refine its suggestions and recommend songs or playlists that match your taste.

The system works by observing your listening habits, identifying patterns, and comparing them to the preferences of other users with similar tastes. 

As you continue to interact with it, the recommendations become more accurate and personalized, making it a powerful tool for discovering music you’ll enjoy.

Key Features: Learning agents are designed to adapt and improve over time. 

They observe their environment, analyze the outcomes of their actions, and adjust their decision-making processes accordingly.

 Unlike fixed-rule systems, learning agents don’t rely on pre-programmed responses—they evolve by learning from experience. 

These agents excel in dynamic environments by refining their strategies over time.

Applications:

• Predictive Analytics: Learning agents analyze large amounts of data to identify patterns and trends. 
  
  For example, they can predict customer behavior, market shifts, or equipment failures, helping businesses make better decisions.

• Virtual Assistants: Systems like Siri, Alexa, and Google Assistant use learning agents to understand user preferences and improve responses over time. 
  u
  They adapt to individual needs, offering more relevant and personalised assistance.

• Dynamic Task Automation: Learning agents excel in automating complex tasks that require flexibility. 
  
  For instance, in logistics, they optimise delivery routes by learning traffic patterns and other real-time variables, improving efficiency and saving resources.
  
  These agents, alongside simpler types, illustrate how AI is seamlessly blending into our lives to address challenges and uncover possibilities

‍Business solutions with AI Agents

AI agents are already part of many business systems. They help teams act faster and reduce manual work.

Sales and marketing: Score leads, personalise campaigns, and track customer intent.
Operations: Forecast demand, plan supply chains, and reroute deliveries.
Customer Service: Handle tickets, learn from feedback, and escalate issues when needed.
Data and Analytics: Organise data, find insights, and assist decision-making.

These examples show how AI agents move beyond experiments into measurable outcomes — much like the work done in AI-driven Drupal implementations, where smart automation and structured reasoning simplify everyday workflows.

The same approach also extends to content optimisation, where systems can automate SEO in Drupal by analysing intent and improving page performance over time.

‍Business solutions with AI Agents

When applied correctly, AI agents deliver real business gains.

• Higher productivity: Routine work happens automatically.
• Lower costs: Repetitive processes need less human oversight.
• Easier scale: Agents can support multiple workflows once trained.
• Personalisation: Users get services that fit their needs.

These results explain why many teams are now turning to agentic systems as a strategy for long-term growth, systems that deliver visible ROI by scaling intelligent processes across teams.

Challenges and considerations

Like any system, AI agents come with challenges.

• Human oversight: Agents need people to guide and review actions.
• Data quality: Results depend on reliable, structured data.
• Privacy and compliance: Information must be handled responsibly.
  ‍‍

How to get started

For teams ready to begin, start simple and scale with results.

1. Find the right use case: Choose a workflow that repeats often and requires decisions.
2. Check your data: The cleaner the data, the better the agent performs.
3. Work with partners: Collaborate with experts who understand both tech and business context.

‍

What’s next

The types of AI agents, from simple reflex agents to sophisticated learning agents, highlight how these systems integrate into our daily routines and industries

Countries like the United States, China, and European nations are leading in AI agent development.

The U.S. is home to tech giants like Google, OpenAI, and Microsoft, driving groundbreaking innovations such as GPT-4 and Google's Gemini. 

In China, companies like Alibaba and Tencent are making significant strides, even amid challenges like chip export restrictions. 

European startups are carving their niche, focusing on valuable applications of AI to drive industry-specific solutions.

The next frontier involves developing more autonomous and sophisticated AI agents capable of advanced reasoning and decision-making. 

At CES 2025, Nvidia CEO Jensen Huang revealed exciting updates in AI and computing. The new GeForce RTX 50 Series GPUs, built on the Blackwell architecture, bring faster and smarter graphics for gamers and creators. 

Nvidia introduced Project Digits, a $3,000 personal AI supercomputer that lets researchers and students run large AI models at home instead of relying on cloud services. Another major highlight was Cosmos, a set of AI models designed to help robots and self-driving cars learn from realistic videos. 

Nvidia also announced partnerships with companies like Toyota to improve autonomous vehicle technology. These updates show Nvidia’s focus on leading innovation in AI, robotics, and gaming.

While AI agents promise to address global challenges like healthcare access and education, their integration into daily life requires careful consideration to ensure they enhance human lives without unintended consequences.

From simplifying schedules with smart assistants to transforming healthcare and logistics.

AI agents are moving from concept to practical business tools. They help organizations act faster, serve better, and keep improving.

The future of automation lies in systems that think and adapt but stay guided by human goals.

FAQs

What’s the difference between a chatbot and an AI agent?
Chatbots follow scripts. Agents plan and act to achieve goals.

Can agents learn on their own?
Yes, they improve by observing outcomes and adjusting their approach.

What tools are used to build agents?
Frameworks like LangChain and CrewAI connect reasoning models with everyday business tools.

‍

‍

Amongst all the other things that keep us up at night, one of them has to be the effect that AI has over us and how it’s becoming a significant part of our decision-making, influencing everything from loan approvals to job recruitment. 

The global AI market, expected to reach $1.8 trillion by 2030, offers ample opportunities but also raises ethical questions.

Generative AI (GenAI) is a type of AI that creates new content, like text, images, music, or videos, by learning from large amounts of existing data.

While this makes GenAI very useful in many fields, it also comes with challenges like creating unfair or biased content, sharing private information, or being a black box without any way of knowing how it ended up generating something"

In this blog, we will see how to approach these challenges responsibly so that GenAI is beneficial and inclusive for everyone.

‍

‍

Privacy

Generative AI uses huge amounts of data to create customized results, like suggestions or content. 

But using so much data brings important questions about keeping it safe, getting proper permission, and protecting it from misuse.

Data security: As of 2023, 83% of companies reported experiencing data breaches, highlighting the need for stronger protections. 

Sensitive information, like health records and shopping histories, needs to be protected. 

Companies using generative AI can take steps to keep data safe and build trust. This includes locking sensitive data with strong encryption, regularly checking for security risks, and limiting access to only those who need it. 

Consent and transparency: Research shows that 72% of users are unaware of how their data is collected or used. 

Bias

AI systems learn from past data, which often reflects societal biases. If these biases aren't dealt with, they can result in unfair decisions in areas like hiring, healthcare, law enforcement etc.

Training data bias: A 2019 study found that AI recruiting tools favored male candidates over female candidates because they were trained on biased data. Regular checks and using a wide range of data can help reduce these problems. 

Representation bias: Facial recognition systems, for instance, have an error rate of 34% for darker-skinned women compared to just 1% for lighter-skinned men. 

This gap highlights the importance of using fair and diverse data for training can help reduce these gaps and make AI systems more equitable.

Transparency

The challenge with GenAI is that it works like a “black box” – people don’t always know how it makes decisions. This lack of clarity can make it hard to trust the results or hold anyone responsible when something goes wrong.

Trust and clarity: A 2022 survey revealed that 60% of consumers are hesitant to use AI-driven systems because they don’t understand how decisions are made. 

AI systems need to clearly explain their decisions, like why they approved a loan or suggested medical treatment, in a way that’s easy for anyone to understand.

Ethical oversight: In important areas like healthcare and criminal justice, being clear is a must. When decisions are easy to understand, it’s simpler to spot mistakes or unfairness and hold organizations responsible for them.

‍

‍

Navigating ethical challenges with practical solutions

To build fair and trustworthy AI, businesses need to focus on three key areas:

Protecting privacy

People are concerned about how their personal information is used. To build trust, businesses need to handle data carefully.

Keep data anonymous: Remove personal details from data to protect people’s identities while still making the information useful. 

For example, streaming services can analyze viewing habits without revealing personal accounts.

Use privacy techniques: Differential privacy is a way to protect individual data by adding random changes, or noise, to it. This lets companies find trends without revealing personal details. 

Apple uses this approach to improve features like QuickType suggestions and emoji insights while keeping user data private.

Follow data laws: Laws like GDPR in Europe or CCPA in California require businesses to safeguard personal data. 

Meeting these standards not only protects people but also builds customer trust.

Reducing bias

AI systems often reflect the biases in the data they’re trained on, which can lead to unfair outcomes.

Use Diverse data: Train AI on data that includes people of different backgrounds, ages, and experiences. 

For example, adding accents and regional dialects improves speech recognition for all users.

Check for fairness regularly: Businesses should test their systems frequently to catch and fix unfair outcomes. 

Tools like Google’s What-If Tool make it easier to spot problems.

Build inclusive teams: Involve people from different backgrounds in AI development. A diverse team can help identify and solve bias issues early on.

Improving transparency

When people don’t understand how AI works, they’re less likely to trust it. A study by Deloitte found that 62% of consumers want AI systems to explain their decisions clearly.

Make AI explainable: Build systems that show how decisions are made in simple terms. 

For example, some AI tools include features that let users see why certain outcomes were chosen.

Open up where possible: Share parts of your AI systems, like algorithms or training data, with trusted experts. 

This adds credibility and allows others to confirm the system is fair.

Help people understand: Offer clear guides or examples that show users how your AI works and how their data is used. This makes them feel more in control.

‍

The future of ethical GenAI

Generative AI is already reshaping industries by tackling complex challenges, creating new opportunities, and delivering meaningful value across sectors like healthcare, finance, energy, and the creative economy. 

In healthcare, AI-driven tools already analyze medical images with up to 90% accuracy, improving diagnostics and treatment outcomes. 

Financial systems use AI to reduce fraud, a global issue costing businesses $5.38 trillion in 2023, while in energy, AI is accelerating breakthroughs in sustainable technologies like next-generation batteries. 

The global generative AI market is projected to grow from $10.63 billion in 2022 to $200.73 billion by 2032, with applications spanning fraud detection, operational efficiency, and creative content. 

Yet, for GenAI to truly serve its purpose, we must confront the ethical challenges it presents. 

Safeguarding privacy ensures trust in systems that handle sensitive data. Tackling bias creates fairness, ensuring AI-driven decisions don't reinforce harmful stereotypes.  

Transparency builds accountability, helping users and businesses understand and rely on AI systems with confidence.

Companies that focus on privacy, fairness, and transparency, won’t just lead their industries; they will set the standard for technology that supports humanity.

As a front-end developer, I have seen a lot of changes this year. A dramatic shift occurred with the arrival of generative AI (GenAI). This shift impacted both clients and developers. 

On the client side, expectations have risen—they now demand more efficient and faster project deliveries. 

On the developer side, we are adopting AI-assisted solutions that enhance our workflows and enable us to complete tasks more efficiently with best practices followed than ever before.

In this blog, I will share my views on the challenges we faced regarding the uncertainty in the front-end (FE) space and the developments that excite me for the near future.

Challenges and solutions in the Front-End workflow

Front-end development traditionally follows a predictable framework: reviewing designs in Figma, translating those designs into HTML and CSS, and refining the output to match project standards. 

This process can be time-consuming, especially when working on large-scale projects.

To address this, GenAI solutions have emerged as game-changers. 

One notable approach is generating markup and styling directly from Figma designs. This automation can save considerable time and make the development process more efficient.

Previously, approaches like AutoHTML and Figma to Code have offered partial solutions, often producing:

• Markup with undesired class names.
• CSS that did not adhere to modern standards or project-specific conventions.

Deep dive: Kombai

Amongst all the available options, Kombai is a revolutionary solution for front-end developers. 

Kombai focuses on converting Figma designs into clean, structured front-end code with minimal manual intervention. 

Here are the key highlights:

• Meaningful class names: Unlike earlier implementations, Kombai generates semantic HTML with meaningful class names, reducing the need for developers to restructure the markup.
• Efficient CSS generation: The CSS output adheres to best practices and modern standards, saving time spent on manual adjustments.
• Customization ready: While the output is clean, developers can further refine the code to match specific project requirements, such as converting CSS to SCSS or replacing static values with design tokens like variables for colors and fonts.

Using Kombai can save developers 30-50% of the time typically spent translating designs to code. 

For example, if a task usually takes 8 hours, Kombai can reduce it to 4-5.5 hours, freeing up valuable time for testing and refinement.

Brief overview of other solutions

While Kombai excels at Figma-to-code conversion, other solutions also bring unique efficiencies, including framework-specific code generation:

• CodeWP: Designed for WordPress projects, it generates custom themes and plugins, providing code tailored specifically for WordPress. It cuts development time by up to 40%. CodeWP can reduce a task that typically takes 10 hours to 6 hours.
• TeleportHQ: A collaborative platform that supports frameworks like React, Vue, and Angular. It enables design-to-code workflows in minutes rather than hours, saving up to 50% of the time. For example, an 8-hour task can be reduced to 4 hours.
• Anima: Generates React, Vue, or plain HTML/CSS code directly from Figma or Sketch. This framework-specific approach can save up to 30% of development time. For instance, a 6-hour task can be cut down to 4 hours.
• Builder.io: Offers AI-assisted no-code/low-code components for frameworks like React and Angular, significantly reducing the time needed for prototyping and development. Tasks that take 8 hours can be completed in around 5-6 hours.‍
• Locofy.ai: Converts designs into responsive code for frameworks like React, Next.js, and Vue. It can cut development cycles by 25-40%, reducing a 12-hour workflow to 7-9 hours.

What’s next for Front-End development?

Looking ahead, GenAI is  set to become even more advanced, enabling:

• Real-time collaboration between design and development teams: solutions will likely evolve to provide live code previews as designs are updated.
• AI-driven code optimization: Beyond generating code, AI will refine and optimize it for performance and accessibility.
• End-to-end automation: From design to deployment, future implementations could automate the entire workflow, reducing repetitive tasks and allowing developers to focus on innovation.

These advancements will save time and enable front-end developers to focus on solving complex problems and creating exceptional user experiences. 

The future of front-end development is fascinating, and with the right expertise and solutions, we can expect workflows to become more streamlined, collaborative, and efficient than ever before.

Stay tuned for more insights into GenAI solutions and their impact on front-end development workflows.

Let's explore how these innovations can continue to make our work faster and more enjoyable!

A self-driving car moves effortlessly through busy streets, noticing traffic lights, pedestrians, and everything around it. It processes all this in real time, deciding when to stop, turn, or speed up, just like a human would. 

Similarly, an AI agent taking care of your everyday tasks. Let’s say it’s booking your flights. It looks at your calendar to figure out the dates, remembers your preferences for seats and flight times, finds the best options, handles the payment, and books everything for you, all without you lifting a finger.

This is the essence of an intelligent agent: systems that observe, think, and act independently to achieve goals. From Siri answering questions to Netflix recommending your next favorite show, these agents seamlessly integrate into our daily lives. 

Thermostats adjusting room temperature, or robots managing supply chains, intelligent agents bridge the gap between automation and intelligence.

In the business world, AI agents handle repetitive tasks, freeing teams to focus on what truly drives progress, acting as reliable partners, uncovering insights in seconds that humans might miss. 

In this blog, we’ll see how they help businesses move faster.

History 

The development of AI agents began in the 1950s and 1960s when pioneers like John McCarthy and Alan Newell created systems that could reason and make decisions. 

In the 1980s, expert systems emerged, using predefined rules to mimic human decision-making. 

By the 1990s, research focused on intelligent agents that could act autonomously, leading to multi-agent systems where agents worked together to achieve shared goals. 

From the 2000s onward, advancements in machine learning and increased computing power enabled AI agents to improve their ability to perceive, reason, and learn, allowing them to operate effectively in dynamic and complex environments.

By 2024, AI agents automate complex tasks and enhance workflows with improved reasoning and autonomy. Companies like Google are deploying AI agents to tackle complex problems effectively.

What are AI Agents?

AI agents are intelligent systems designed to operate independently, taking on open-ended tasks that require planning, decision-making, and adaptation. 

They can gather information, make choices, and perform actions, all while working towards loosely defined goals and learning over time, as well as adapting to their environment, which makes them capable of solving complex, real-world problems.

This ability to sense, think, act, and learn sets AI agents apart. 

They enable businesses to automate complex processes and achieve results with minimal human intervention.

Why are they called Agents and not tools?

The term “agent” reflects their autonomy and decision-making abilities. Tools are passive—they rely on humans to operate and make decisions.

AI agents actively observe their environment, make independent decisions, and act to achieve specific goals which enables continuous operation and adaptability without constant human checks. Calling them agents emphasizes their role as partners in achieving outcomes.

How AI Agents work

AI agents operate in a simple, repeatable cycle to deliver value: they start by understanding their environment, analyze data to decide the best action, carry out that action, and then learn from the results to get better over time.

For instance, they might gather information from customer queries or sensors, use advanced models to predict outcomes, take actions like answering questions or automating workflows, and then improve by learning from what worked and what didn’t.

1. Perception: understanding the environment‍

 AI agents collect information from various sources, such as customer queries, databases, or IoT sensors on factory floors. They interpret this data to understand what’s happening in real time.

2. Decision-making: finding the best next step

With advanced algorithms and models, AI agents analyze patterns, predict outcomes, and determine the best action. For example, a retail AI might forecast demand and adjust stock levels.

3. Action execution: doing the work

Based on their analysis, AI agents step in to act—whether answering a question, automating schedules, or adjusting factory equipment to prevent downtime.

4. Learning: Improving Over Time‍

 AI agents learn from every action and result. They improve their ability to spot patterns, adapt to new challenges, and deliver more accurate outcomes with every cycle.

Why AI Agents matter for growing businesses

The real strength of AI agents lies in their ability to simplify operations, automate workflows, and deliver accurate results. 

For growing businesses, this means less time spent troubleshooting and more time innovating. Teams can make smarter decisions faster, solve problems efficiently, and scale operations without added complexity.

Businesses already using AI agents are seeing measurable results: reduced costs, better resource allocation, and improved customer experiences. As systems learn and adapt, their value only increases over time.

Real-world business use cases

AI agents are increasingly utilized across various industries to address specific challenges and drive business growth. Here are some real-world applications:

Customer service

AI-powered chatbots and virtual assistants provide instant responses to customer inquiries, automating routine tasks and allowing human agents to focus on more complex issues. 

For example, AI agents can handle tasks such as password resets, order tracking, and basic troubleshooting, enhancing efficiency and customer satisfaction. 

Healthcare

In the healthcare sector, AI agents analyze patient data to predict critical conditions, enabling early interventions. 

AI-driven diagnostic tools can detect diseases like cancer and cardiovascular conditions by analyzing medical images and patient records, improving patient outcomes. 

Finance

Financial institutions employ AI agents for real-time fraud detection by monitoring transactions and identifying suspicious activities. 

For instance, AI systems analyze transaction patterns and user behavior to prevent unauthorized transactions, enhance security and reduce financial losses.

Manufacturing

In manufacturing, AI agents utilize predictive maintenance by analyzing sensor data to foresee equipment failures, allowing for timely interventions that reduce downtime and maintenance costs. This proactive approach ensures continuous production and operational efficiency. 

Cybersecurity

AI-driven cybersecurity systems continuously monitor network traffic to detect and neutralize threats before they can cause harm. 

These systems can identify anomalies and potential security breaches, providing a robust defence against cyberattacks. 

Retail

Retailers leverage AI agents to analyze sales data, predict consumer demand, and manage inventory effectively. By forecasting trends, AI helps maintain optimal stock levels, reducing instances of stockouts and lost sales, thereby enhancing customer satisfaction and profitability.

These applications demonstrate the significant impact of AI agents in enhancing efficiency, decision-making, and customer experiences across various sectors.

Future

AI agents are transforming industries in 2024. 

Companies like Telstra, Bunnings, and major banks in Australia are using AI solutions to improve customer service, boost productivity, and streamline operations. Telstra’s AI tools, Bunnings' "Ask Lionel," and Macquarie AI Chat are just a few examples of this shift. 

Banks like ANZ and NAB are adopting AI to save time and enhance workflows, while resource companies like Rio and BHP are focusing on efficiency and sustainability. 

Looking ahead, the global AI agents market is projected to grow significantly. Estimates suggest that the market size will expand from $5.1 billion in 2024 to a remarkable $47.1 billion by 2030, reflecting a Compound Annual Growth Rate (CAGR) of 44.8% from 2024 to 2030.

As AI agents continue to evolve, they are set to become integral to business operations, driving efficiency and growth across various sectors.

Conclusion

For years, the idea of systems working alongside people to handle business tasks seemed out of reach. The real challenge was creating technology that could take on responsibilities independently and reliably.

Today, this is changing. Just as apps revolutionized smartphones, AI Agents are transforming how businesses get things done. With better understanding and memory, they simplify workflows and make processes faster. 

They handle repetitive tasks, analyze data quickly, and offer insights to guide decisions. By fitting into existing workflows, they make processes smoother and help teams get more done.

In the future, these agents will do even more like- anticipating needs, suggesting strategies, and working seamlessly across systems. This allows businesses to focus on creativity and big-picture goals while routine tasks run in the background.

Designing an AI Digital Experience Platform (AI DXP) starts with understanding how people engage with technology in their everyday workflows. It requires clear, functional solutions that integrate AI without complicating the experience.

This blog shares the journey of creating a sector-agnostic AI DXP focused on delivering intuitive, AI-powered solutions. At its core, the platform emphasizes user-centric design—making sure it works seamlessly across industries while adapting to specific needs.

The process was all about answering practical questions — How can AI simplify tasks without taking control away from users? What design choices make the platform feel approachable and effective?

The goal was to create something that feels natural to use while making the most of the tech to improve productivity and decision-making. By focusing on user journeys, we ensured the platform prioritizes usability, making the technology work for people in a way that’s simple, practical, and meaningful.

Building the foundation

To lay the groundwork, we organized a couple of workshops involving stakeholders, project managers, engineers, and business leaders. These sessions helped align user priorities and product objectives, ensuring the platform addressed meaningful challenges effectively.

The design team started by identifying three key use cases and mapping user journeys to establish the foundation of the Minimum Viable Product (MVP). Tasks were approached collaboratively, with regular discussions to integrate ideas. Daily task-level meetings and weekly strategy sessions kept the development dynamic and adaptable, ensuring the platform evolved in line with user and business goals.

This collaborative process made sure that every aspect of the platform was built to serve real-world needs while remaining flexible for future iterations.

A snapshot of the AI DXP Workshop framework, showcasing a collaborative approach to defining project goals, ideating solutions, and prioritizing tasks.

Mapping the user journey — a step-by-step approach

Cross-Functional team collaboration on user journey mapping, This visual captures the detailed user journey map, broken into key stages: Onboarding, Integration, Configuring, Stimulation/Testing, and Analysis. Each column outlines user steps, actions, goals, experiences, feelings, pain points, and opportunities
‍

‍

We identified five key steps that define the user journey for AI DXP.

1. Onboarding

Users begin by creating an account and exploring the platform’s features. This phase includes interactive tutorials and resources that guide users through uploading and integrating their data. By the end of onboarding, users gain a clear understanding of the platform’s capabilities.

2. Integration

Users connect their existing databases, CMS, or external systems to the platform. This step enables the AI to process datasets, providing the foundation for delivering relevant results and actionable insights. Compatibility with popular platforms and APIs ensures smooth integration.

3. Configuration

This phase involves tailoring the platform to specific business needs. Users select the AI features they want to activate, Customised search parameters & some user experience tweaks. These adjustments ensure the AI aligns with their goals.

4. Testing

Users conduct tests to see how the system performs in real scenarios. This includes running sample queries, analyzing how results are displayed, and fine-tuning configurations to improve precision and usability before full deployment.

5. Analysis and Reporting‍‍

Dashboards provide insights into key metrics such as search accuracy, user engagement, and content performance. Users can review analytics like common queries, areas of high engagement, and system-generated suggestions. These reports help track performance and highlight areas for optimization.

Each step was mapped to specific user interactions, emotions, potential friction points, and opportunities for improvement. This granular approach ensured the platform delivered a streamlined experience while addressing real challenges users face in managing and interpreting their data.

User flow and workflows 

Based on the user journey, we identified three essential workflows to focus on for the Minimum Viable Product (MVP). 

‍

Prioritized user flow defined after team alignment for MVP, A user flow diagram illustrating the step-by-step journey users take through the product. This visual maps out interactions, decisions, and transitions, providing clarity on pathways and potential bottlenecks.

Onboarding

Getting users started and comfortable with the platform.

Onboarding was all about making the first steps clear and approachable. We worked on simplifying account setup and guiding users through their initial interactions with the platform. A lot of attention went into answering questions like:

• How can users see the platform’s potential right away?
• What’s the simplest way to help them get started?
• How do we make this first step feel smooth and purposeful?

The result was an onboarding experience that was straightforward and practical, helping users feel confident as they began exploring the platform.

Configuration

Letting users set up the platform to match their needs.

This workflow focused on creating a simple, clear process for users to adjust the platform’s settings and features based on their goals. Whether it involved customizing preferences or adjusting workflows, the emphasis was on ensuring flexibility without overwhelming complexity.

Testing

Helping users fine-tune the platform with hands-on feedback.

Testing allowed users to explore how the platform worked in their context. Through small iterations and adjustments, they could refine its functionality to fit their specific needs. By testing features in real time, users were able to see what worked and adjust anything that didn’t.

These workflows formed the foundation of the MVP, ensuring the platform was functional, easy to adopt, and adaptable to different use cases without over-complicating the experience.

Building the core

At the centre of this product is an AI-powered content management system (CMS) designed to simplify workflows and automate complex processes. AI capabilities were integrated to handle tasks like dynamic content generation, data management, and personalized content delivery, reducing the need for manual intervention.

The platform includes AI-driven features for content tagging, automatic metadata generation, and contextual suggestions to improve usability and save time. These tools enable the CMS to adapt to user behaviors, making it intuitive and practical for day-to-day operations.

The focus remained on ensuring the system delivers value to users by automating repetitive tasks and enabling smoother interactions with content and the workflows they want to set up.

Guiding principles for good UX

When designing the platform, we focused on two key principles to make the experience intuitive and effective.

1.User control

We wanted users to feel confident and in charge of how they use the platform. Whether it’s deciding when to activate AI integrations or how they fit into their workflows, the goal was to give them clear control without unnecessary complexity.

2.Error prevention and iteration‍‍

We built the platform to allow users to test things as they go. They can tweak configurations, see immediate results, and make adjustments in real-time. This process helps them refine how AI fits into their needs without worrying about getting things perfect the first time.

These principles guided every design decision, ensuring the platform feels approachable and adaptable. By keeping users at the center, we created something that works across different industries while still feeling personal and practical.

‍

Lessons learned 

Designing AI products is a constant exercise in balancing complexity, uncertainty and simplicity. There’s always a tension between pushing the limits of what technology can do and making sure it still feels approachable for the people using it. Along the way, we learned that the best solutions come from listening—whether it’s through workshops, team collaboration, or iterating based on feedback.

One of the biggest takeaways was the importance of focusing on what truly matters: flexibility, reducing friction, and giving users the control they need to trust the platform. It wasn’t about cramming in features but about creating something that fits naturally into their workflows and makes their lives easier.

Looking ahead, these lessons will guide how we approach every new challenge. Whether we’re rethinking a small detail or introducing entirely new features, the goal will always be the same: to build AI products that feel intuitive, adapt to real-world needs, and make a tangible difference for the people using them. This journey with AI DXP has set the tone, and we’re excited to see where it takes us next.

Everyone with a phone in hand, or even just a thought in mind, is always searching for something—whether it’s a recipe, directions to a new café, or the answer to a work question. 

Searching has become second nature. But finding the right answer? That’s a different story.

This is where AI search assistants step in.

Instead of scrolling through endless links or sifting through irrelevant results, they interpret intent, adapt to your needs, and deliver answers that are clear, accurate, and actionable.

For years, AI has been part of our lives, recommending videos, optimizing photos, and suggesting replies. But now, AI truly understands. Search is something we all rely on, whether out of curiosity or for everyday tasks.

‍In this blog, we’ll discuss the need for faster, more accurate, and contextual search assistants, how they improve efficiency and real-world use cases. We’ll also touch on privacy and security concerns and explore what the future holds for AI search technology

What is an AI search assistant?

An AI search assistant is a system powered by artificial intelligence that helps users find accurate and relevant information quickly. By understanding the intent behind your queries, it provides precise, actionable answers without relying on basic keyword matching. 

Using natural language processing, AI search assistants can interpret complex questions and deliver responses tailored to your needs, making searches more efficient and intuitive.

This technology streamlines the process of finding information, saving time and helping users access the data they need faster, whether for personal or professional use.

It figures out the meaning behind your questions and finds the most useful answers, like a search that actually gets you.

The importance of AI search assistants in business

Businesses have a clear challenge: getting the right information when they need it most. 

Whether it’s a doctor searching for clinical protocols or a lawyer looking up case laws, a lot of time is wasted going through scattered, irrelevant results.

The problem isn’t about having enough data, there’s plenty of that. It’s about making information work smarter. 

Search systems need to understand the intent behind a query, recognize the industry-specific language, and deliver answers that are clear, accurate, and useful.

As organizations grow, so does their data, spread across various platforms. What should be a quick search turns into an endless task, draining time and slowing down decisions.

The bottom line? 

People need information that finds them, not the other way around.

AI-assisted search solutions

AI search assistants overcome these limitations by offering intelligent, context-aware solutions tailored to specific needs:

Semantic search

Semantic search understands the meaning behind your query. Instead of matching exact keywords, it considers intent and context. For example, searching “treatment options for chronic back pain” identifies therapies, medications, and specialists for long-term pain management. 

It works by using natural language processing (NLP) and machine learning to provide accurate and meaningful results.

Image search

‍
Image search uses AI to analyze visual content, shapes, colors, and patterns, to deliver precise results. For instance, searching “green running shoes” brings up images of shoes that match the color and category. 

It helps users find products, designs, or references faster by understanding what’s in an image.

Predictive search

‍
Predictive search suggests results as you type by learning from past searches and behaviors. For example, if you often search for “vegan recipes,” typing “vegan b…” might show “vegan brownies” instantly. 

This saves time and anticipates what users need.

Personalized search

‍
Personalized search delivers results tailored to your preferences and history. If you’ve been shopping for “winter jackets” on a retail site, it might prioritize jackets in future searches. 

By analyzing activity and behavior, it offers a custom, user-centric experience.

Federated search

‍
Federated search gathers results from multiple sources into one organized list. For example, in an enterprise system, searching “quarterly sales report” might pull data from emails, databases, and shared folders, saving time and centralizing information.

Knowledge graph search

‍
Knowledge graph search links related information to uncover deeper insights. Search “Leonardo da Vinci” and you’ll see his works, inventions, contemporaries, and historical context, all connected.

 It helps users explore relationships between data points and gain a broader understanding.

Natural language search

Natural language search allows users to search conversationally. For example, asking “Where can I buy affordable hiking gear?” returns relevant stores and products without needing exact keywords. 

It works by interpreting intent and understanding human-like queries.

Digital Asset Management (DAM) search

‍
DAM search helps users quickly locate files like images, videos, or documents by analyzing metadata and content. For instance, searching “summer campaign video” in a DAM system surfaces the correct video file by recognizing tags, dates, and descriptions.

It streamlines file management and saves time.

How AI search assistants work

AI search assistants stand out by understanding context and user intent, ensuring every query delivers the most relevant information. Here’s how:

Custom search indexes

Industries can build their search catalogs to meet unique needs. For example, a healthcare company can prioritize clinical guidelines, research papers, and protocols in its search results.

Smart ranking and sorting

AI ranks and organizes results based on what matters most. For instance, a search for “asthma treatment” delivers the latest research and updated guidelines first.

Industry-specific context adaptation

AI understands the relationships between terms in different industries:

• Healthcare: Links symptoms, treatments, and outcomes.
• Legal: Highlights case laws, statutes, and legal documents.
• Education: Focuses on learning materials, curriculum plans, and performance data.

Testing and refining for accuracy

Organizations can test the search system before rolling it out. This ensures results are precise and meet expectations.

Future ready and scalable design

AI search systems evolve with time. They learn from how people use them and can integrate with new tools or platforms as businesses grow.

Industries that can benefit from AI search assistants

AI search assistants work exceptionally well in industries with complex data needs:

‍

1. Healthcare
   
   AI search tools enable healthcare professionals to swiftly access clinical research, treatment protocols, and patient care guidelines. The NHS has utilized AI to diagnose COVID-19 from chest imaging and assist in dermatology referrals, streamlining diagnostic processes.
   
   

2. Legal
   
   Lawyers benefit from AI by quickly retrieving case laws, statutes, and legal references. The legal industry's technology spending is projected to increase from $26.7 billion in 2024 to over $46 billion by 2030, reflecting a significant investment in AI tools to enhance legal research and workflow automation.
   
   

3. Education
   
   Educators utilize AI to efficiently access curriculum plans, learning materials, and student performance data. AI technologies are increasingly used in clinical education, with 58% of trainee doctors perceiving a positive impact on their training, indicating AI's role in improving educational resources and training efficiency.
   
   

4. Media management
   
   Content creators leverage AI to locate images, videos, and digital assets effectively. The global AI in media and entertainment market was valued at $14.81 billion in 2022 and is expected to reach $99.48 billion by 2030, growing at a compound annual growth rate (CAGR) of 26%. 

‍

AI search assistants are valuable across different industries. In e-commerce, they help users cut through product overload by refining search results based on what they actually need, making it faster and easier to find the desired products.

In nonprofits, AI search assistants simplify the process of finding key information, funding opportunities, or program details, helping users quickly access the resources they need.

In essence, AI search assistants enable businesses—whether large or small—to efficiently access crucial information, such as product details or internal documents. These systems quickly deliver the most relevant results.

Data privacy and security

Keeping data private and secure is essential for businesses using AI search systems, designed to handle sensitive information carefully, ensuring teams can work efficiently without compromising trust. 

Here’s how they make data security simple and reliable:

1. Keeping data separate
AI search systems isolate each team's data, meaning no one outside the team can access or view it. This ensures information stays protected within its own space. According to a report by McKinsey, 88% of organizations believe strong data isolation is critical to preventing leaks and maintaining privacy in collaborative environments.

2. Secure system connections
AI safely connects with existing tools and platforms, reducing risks during data transfers. For example, improperly secured integrations are a common target for cyberattacks, which cost businesses an average of $4.45 million per breach in 2023, as reported by IBM’s Cost of a Data Breach Report.

3. Following global privacy rules
AI search systems meet strict industry standards like GDPR (in Europe) and CCPA (in California). These rules ensure businesses protect user data and avoid fines. Under GDPR, businesses that fail to comply can face penalties as high as 4% of their annual revenue.

4. On premise and cloud deployment
AI search systems offer two deployment options to suit different privacy needs:

• On-Premise: Data is stored and managed on a company’s servers. This gives full control over security and ensures sensitive data stays within the organization. 

It’s ideal for businesses with strict privacy or compliance requirements.

• Cloud: Data is stored on third-party servers(like AWS or Google Cloud) and accessed via the internet. 

Cloud deployment is cost-effective, easy to scale, and managed by trusted providers who follow global security standards.

5. End-to-end encryption
AI search systems use end-to-end encryption to protect data during storage and transfer. 

This ensures that even if data is intercepted, it cannot be read or accessed without the proper keys. 

According to IBM's Cost of a Data Breach Report 2023, organizations using encryption reduce data breach costs by an average of $220,000 per incident, highlighting its role in safeguarding sensitive information.

The future of AI search assistants

AI search assistants are learning to think the way we search, becoming smarter, sharper, and more human in how they respond. 

Soon, searching will feel less like sifting through data and more like having a natural, intuitive conversation, where systems not only understand what we’re asking but anticipate what we need next.

For businesses, this changes everything. The endless hunt for files, scattered information, and missed connections gives way to focus, flow, and clarity. 

Teams can work faster, find answers that matter, and spend their time building, creating, and delivering.

What’s exciting isn’t just the speed or accuracy, it’s the shift from searching for information to using it. 

These systems are becoming quiet partners in the background, helping us move through decisions and actions with ease. 

The right answers, at the right time.

Unfindable and unstructured information becomes a valuable resource when organized and made accessible to the right people at the right time. 

AI-assisted solutions, such as semantic search, intuitive chatbots, and intelligent workflow support, enhance operations by making data navigable and actionable, ensuring businesses and users can access important information when needed. 

AI DXP simplifies the integration of AI into business processes. It offers pre-configured models for specific tasks and flexible settings to adapt to unique requirements. Businesses can customize AI assistants, save time, and reduce costs by reusing existing data indexes. Built-in security ensures chatbots remain protected from misuse.

Ready-to-use SDKs (software development kit) streamline setup, allowing businesses to focus on delivering better digital experiences without being burdened by technical hurdles

This blog explains how AI DXP streamlines AI integration into business workflows, offering solutions like semantic search and chatbot configuration to handle unstructured data effectively, enhance user experiences, and reduce inefficiencies across industries.

What is AI DXP?

AI DXP, or Artificial Intelligence Digital Experience Platform, helps integrate AI into business workflows. Built for practical implementation, it simplifies setup and ensures seamless integration, accommodating teams across varying technical expertise. 

The platform currently supports two main features: Semantic Search, and Chatbot (RAG), both tailored for contextual user interactions. Its future roadmap includes agent-based capabilities to optimize operational workflows and processes, and make them adaptable to evolving business needs and market shifts.

AI DXP offers a practical approach to improving digital workflows. A unified dashboard allows businesses to manage services, configure settings, and test outcomes before deployment, providing clarity and control throughout the process. 

Key features and technology

AI DXP is designed for flexibility and easy customization, allowing businesses to align solutions with their unique needs. Through the initial consultancy phase, we provide expert guidance to help teams define their AI assistant goals, configure workflows, and deploy solutions efficiently.

One of its key features is semantic search, which enables users to find relevant information without relying on exact keywords. The team can customize this by adjusting settings like embedding models, and choosing between third-party or QED42-hosted options. The platform also includes advanced contextual chunking—breaking long content into smaller, meaningful parts while preserving the relationship to the overall content. For simpler tasks, cost-effective chunkers are also available, and teams can preview and adjust results before going live. 

AI DXP allows teams to create custom-configured AI-powered chatbots that integrate seamlessly with the existing platform. With a curated list of AI models, teams can fine-tune chat prompts and set up security guardrails to ensure bots are reliable. Reusable data indexing helps save time and costs by repurposing existing indexes for chatbots. 

The platform’s technology ties everything together. Analytics provide insights into query success and response accuracy, helping businesses refine their solutions. 

Use cases

AI DXP provides benefits for industries that handle and harness large amounts of information, extracting and making the best use of data to support teams and other platform users. Here are some use cases:  

Legal

AI-assisted solutions can help legal professionals find case files and clauses accurately and faster by searching through large document libraries with natural language processing (NLP). Small law firms can upload case files and use AI to quickly find precedents, saving time and making research easier. Lawyers can ask, “Show me cases where a breach of contract was upheld in New York State.” The platform highlights relevant rulings and key arguments. 

Healthcare

‍
Patients can quickly find health information that fits their needs, cutting search time. Hospitals use AI to help medical teams find clinical guidelines faster, making care more efficient. Doctors can ask, “Find me the record of that one patient who had leukaemia last December" and instantly receive accurate, context-aware results. AI DXP’s semantic search interprets queries in natural language, reducing the time spent sifting through documents manually.

Insurance and claims

AI-assisted solutions can help banks, hospitals and insurers quickly search and access reports, policies, or any information they need at the moment. In health insurance, queries like “Does the policy cover pre-existing conditions for diabetes?” retrieve policy-specific insights, reducing manual review time.

‍

Nonprofits

‍
Nonprofits can quickly find reports, grants, and insights, reducing errors. Volunteers also get easy access to training and guidelines when needed. Queries like, “What are the safety protocols for working in disaster zones?” or “How do I log my volunteer hours?” Indexed training manuals provide clear, accurate responses. AI DXP has the potential to help non-profits centralize critical resources, improving efficiency in volunteer onboarding and support.

Looking ahead 

As AI DXP evolves, it will make integrating AI into business workflows more practical and reliable, keeping humans at the core as an important ethical principle.

The upcoming updates and new features in the product roadmap will streamline the process of connecting the platform to existing solutions, making them more contextually relevant for a wide range of industries. This will also ensure faster and more efficient deployment.

Offering seamless on-premise implementation as an option, alongside new capabilities like agent-based automation, personalized content delivery, and real-time analytics, thoughtfully designed to address complex business, digital, and workflow challenges. These features will help teams solve complex tasks, deliver relevant content, and save time, enabling them to focus on leveraging their human expertise.

As AI DXP evolves, the shift towards more plug-and-play solutions will allow businesses to quickly implement AI features with minimal setup. Well, looking ahead, AI DXP is designed to be a practical technological asset that grows alongside the needs of businesses, not just in line with trends. Get a closer look—with a scheduled demo.

Every day, professionals come across an overwhelming amount of information. Reading, analyzing, and reviewing this information is a constant challenge in industries where precision and speed are important.

An AI document assistant addresses these challenges by sifting through diverse content—from text to visuals—identifying key takeaways, integrating with internal systems and platforms like blogs or other content repositories.

This assistant supports teams by harnessing and understanding information based on their precise queries, doubts, and needs of the moment, reducing errors, saving time, and improving the overall operational workflow.

This blog explains how AI Document Assistants help professionals manage complex information by simplifying tasks like retrieving, analyzing, and summarizing content. It also outlines the practical uses and features of QED42's AI document assistant. 

‍

Challenges

Modern workflows demand systems that simplify complex information into actionable insights. Professionals simply need fast, accurate access to critical details across diverse industries, from patient records to financial reports

Users operating within many industry systems require fast and accurate access to critical information, whether it’s patient records, legal documents, financial reports, or academic materials. These users need systems that can handle large volumes of information, integrate seamlessly with their existing platforms, and provide relevant insights when needed to support decision-making.

A 2022 report by Coveo found that employees spend an average of 3.6 hours daily. This further proves that the need for advanced AI-assisted solutions is undeniable. 

By understanding the problem and researching how professionals and users in specific industries work, we created an AI document assistant that’s simple to use, responds quickly, works well with current workflows could be scaled as needed. It helps save time, reduces mistakes, and supports people, especially those who depend on web-based information. 

What is an AI document assistant?

AI document assistants are smart, practical solutions that help people find, understand, and work with information from different types of documents, like PDFs, Word files, and spreadsheets, as well as content like text, tables, and pictures. 

They use advanced technologies like natural language processing (NLP) and vision-language models (VLMs) to simplify how questions are asked and answered. 

NLP enables systems to process and understand human language by analyzing text or speech. It allows AI to interpret questions, commands, or any form of language input accurately. 

VLMs are trained on extensive datasets, enabling them to extract meaning from complex visual and textual elements, even when explicit text is absent. They combine visual and language understanding to handle tasks like interpreting charts, images, or documents with minimal cues.

The AI document assistant, designed and built using these technologies, is for people who deal with large amounts of information, helping them make sense of it quickly and find what they need.

With evaluation systems designed to assess content relevancy, contextual understanding of the language model, and predefined response boundaries with guardrails, they ensure answers are accurate and relevant. 

Designed to meet diverse industry needs, these assistants simplify everyday tasks by adapting to varied document formats and user contexts.

While AI Document Assistants can be powerful, their performance depends on how they are designed and where they are applied. For instance, not all assistants can work effectively with tables or images unless they are specifically built for those tasks. 

Many systems allow conversational interactions, but the usefulness of their answers often depends on the quality of the source data, how well the system is designed, and the clarity of the user’s questions. Some assistants may not include advanced features like reliable fact-checking or deep contextual understanding, as these safeguards vary across implementations.

It's important to make sure the document assistant is built thoughtfully and its capabilities align with the complexity of the tasks it is meant to handle and the specific needs of the industry it serves.

Implementation and practical use 

AI-Doc Assistant is built to address the specific challenges people face when working with extensive and complex content repositories. Its features are designed to make information accessible, actionable, and usable

Enhanced accessibility

For industries like finance, healthcare, legal, and life sciences, where important information is often stored in dense PDFs or unstructured formats, the assistant simplifies the process of finding, extracting, and understanding data. From regulatory filings and clinical trials to legal documents, it helps transform static content into valuable resources.‍

‍Tailored industry applications‍

The assistant’s NLP capabilities are adapted to industry-specific terminology, ensuring it meets the unique needs of professionals in finance (audit reports and compliance), healthcare (clinical guidelines and patient records), or education (research papers and academic material).

Collaborative data insights

When connected to organizational content repositories, the assistant ensures that important information becomes available to teams across functions, supporting cross-functional collaboration and informed decision-making without requiring domain expertise.

The AI document assistant shows its impact across industries—healthcare professionals can retrieve patient records from scanned files, legal teams can locate specific clauses in case files, and financial analysts can identify trends in regulatory filings—all with greater efficiency. 

We are running pilots to develop systems that simplify navigating extensive documents, convert static PDF archives into searchable repositories, and extract actionable insights from complex content. 

These efforts focus on making important information easily accessible across teams and other users as needed, whether for compliance, reporting, strategic planning, operational execution, or collaboration on shared goals. This ensures organizations can streamline workflows, address bottlenecks, and make better use of their data.

‍

Why it works

As AI document assistants are designed and built to handle complex document tasks with accuracy, adaptability, and speed, their strength lies in providing clear, meaningful results that align with user needs. The main reason why this solution works is:

1. Contextual understanding
   It processes data in a way that aligns with the meaning and intent behind user queries. This ensures the information provided is relevant and precise, even when working with detailed or unstructured content.
   
   
2. Flexibility
   It supports a variety of formats, including PDFs, Word documents, handwritten notes, and scanned images, making them suitable for content of every shape, size and type across industries.
   
   
3. Efficiency
   By automating essential tasks like summarizing documents, identifying key details, and organizing information quickly and accurately, these assistants allow users to focus on decisions and actions. Anyone who works with information daily understands the importance of this.

Those who work with information daily understand how much these capabilities—contextual understanding, flexibility, and efficiency—shape their ability to handle diverse tasks and respond to unexpected demands. The time saved can be redirected toward strategic initiatives, creative problem identification and solving, or addressing high-priority challenges.

‍

Potential

The AI Document Assistant and information-harnessing solutions change how people interact with information; here is data from surveys and reports illustrating their impact across industries.

In healthcare, AI Document Assistants can help manage administrative tasks, which often take up to 30% of a professional's time, allowing more focus on patient care and clinical priorities.

In finance, they can assist with the analysis of audit reports and regulatory filings. McKinsey reports that AI solutions may reduce document processing times by 45–50% and improve accuracy by over 30%, helping professionals identify trends, risks, and opportunities more effectively.

In education and research, AI solutions can filter and summarize dense academic materials, with PwC studies indicating productivity gains of up to 25%.

Our research and understanding of the legal industry suggest that AI-assisted solutions can help streamline the review of contracts, compliance documents, and case files, with additional use cases continuing to emerge.

According to the American Bar Association, over 60% of legal professionals spend substantial time on these tasks. AI tools have the potential to improve efficiency by up to 40%, freeing up time for strategic work and client interactions.

The true value of these solutions depends on how thoughtfully they are used. 

The real question is not what the technology can do, but how it can be shaped to serve genuine human needs. 

A good approach is focused on understanding the specific challenges and workflows in which AI Document Assistants and similar solutions can be applied.

Engaging directly with users to identify pain points, testing solutions in real-world scenarios, and iterating based on feedback are critical steps.

The next steps should prioritize collaboration between technology providers and end users to bridge the gap between potential and practicality.

This ensures the technology is functional, genuinely useful, intuitive, and aligned with the goals of those who rely on the platform and use the solution.

Looking ahead 

The changes coming in with artificial intelligence, guided by human imagination, are redefining how we access, harness, and interact with information. 

Whether we call them assistants, solutions, or agents, the goal remains the same—to support people (us humans) in navigating complexity with ease and make our lives simpler (professional and personal).

AI Document Assistants, including QED42's solution, help professionals and other users manage information efficiently and adapt to the demands of an information-rich environment. 

When thoughtfully implemented, This and other similar solutions have the potential to become our partners—enablers for people to focus on collaborating effectively with clients and colleagues, ensuring precision in tasks that demand human expertise, and dedicating time to priorities efforts that contribute to both personal growth and team success.

We are very used to a world where the smartest thing on the planet is us. And we are, either wisely or not, changing that. We are building something smarter than us, way smarter than us.

Nowhere is this shift more powerful than in healthcare. AI is still finding its way here, but people are all in because it brings real promise and hope for a healthier future.

AI can provide health advice, which is especially important since doctors are in short supply — even in wealthy countries that invest heavily in healthcare. For instance, the United States faces a projected shortage of up to 124,000 physicians by 2034. As you move into poorer countries, where healthcare resources are even more limited, many people may never see a doctor in their entire life. In sub-Saharan Africa, for example, there are often fewer than 0.2 doctors per 1,000 people, leaving many reliant on community health workers for basic medical needs. 

For example, Sybil—is an AI model developed by researchers at MIT to assess the risk of lung cancer by analyzing CT scan images. Unlike traditional methods, Sybil doesn't rely on radiologists to flag cancerous signs, instead, it predicts a patient's likelihood of developing lung cancer within six years, even in the absence of visible signs of the disease. This model has shown strong predictive power, helping doctors identify patients who may develop lung cancer before it becomes detectable, potentially saving lives through earlier intervention.

Similar to this technology and others currently being researched and developed, AI addresses very complex problems. We're still living in a world of scarcity, with a shortage of doctors. The fact that AI is set to accelerate advancements in these areas is definitely something worth discussing—and perhaps even celebrating.

Let’s explore why it is a genuine need in the healthcare industry.

The growing financial burden

One of the biggest challenges is the rising cost of healthcare. As more people develop chronic conditions like diabetes and heart disease, they require ongoing care, especially as they age. Coupled with the rising costs of treating these conditions, the financial burden continues to grow, adding strain to both individuals and healthcare systems.

How AI can help

AI-assisted diagnostics can help reduce costs by catching diseases earlier, allowing for less invasive and more affordable treatments. By providing precise diagnoses at an early stage, tech could help patients avoid multiple tests and expensive procedures, leading to better outcomes without adding unnecessary financial strain. 

For example, AI-driven predictive analytics can help identify at-risk patients and facilitate preventive measures, ultimately lowering overall healthcare expenses.

Tools like IBM Watson and Google Health's AI models are being used to analyze patient data, identify patterns, and predict potential health risks before they become critical. By integrating data from electronic health records, wearable devices, and other sources, these AI tools provide actionable insights that enable healthcare providers to offer targeted interventions, personalized care plans, and early lifestyle recommendations, thereby reducing the long-term costs associated with chronic conditions.

Scarcity of doctors 

As the global population continues to grow and age, there are simply not enough healthcare professionals to meet the rising demand. The U.S. alone is projected to face a shortage of between 37,800 and 124,000 physicians, with both primary and speciality care sectors affected.
Patients often face longer wait times, and healthcare providers are overloaded with cases, limiting the attention they can give to each individual patient.

How AI can help

AI virtual assistants and chatbots can take on routine tasks such as scheduling appointments, answering common patient questions, and monitoring chronic conditions. 

This frees up doctors to focus on more complex cases, reducing their workload and the risk of burnout while still ensuring timely care for all patients. 

AI-driven search tools make patient data immediately accessible, allowing healthcare professionals to make faster, more informed decisions. 

Tools like Nuance's DAX (Dragon Ambient eXperience) and Babylon Health's AI assistant are already being used to streamline administrative tasks and provide real-time support, enabling doctors to dedicate more time to direct patient care and complex decision-making.

Putting data overload to use 

Healthcare providers generate large volumes of data, including patient records and research findings, but much of it goes unused because healthcare professionals often lack the time and resources to fully process it. In fact, studies estimate that approximately 97% of healthcare imaging data goes unused, leaving a significant opportunity for technology to help make sense of it all.

How AI can make it better

AI-based solutions are transforming data usage in healthcare. AI tools analyze large datasets, identifying critical insights and supporting faster decision-making. 

This has proven valuable for diagnostics—Scientists at Harvard Medical School have developed a versatile AI model similar to ChatGPT, designed to perform a range of diagnostic tasks across various types of cancers. This AI system, named CHIEF, is capable of detecting cancer, predicting tumour genetics, and estimating patient survival based on histopathology images. The model was trained on millions of images from diverse cancer types and has been shown to outperform existing AI methods in accuracy, making it a promising tool for supporting clinical decision-making in cancer care.

AI-assisted solutions like these provide real-time synthesis of information, giving clinicians immediate access to relevant and reliable information. This supports decision-making, reduces administrative tasks, and allows the healthcare workforce to focus more on patient care. 

Some real-world applications

1.AiCure‍

AiCure uses advanced AI algorithms to monitor patient behavior and confirm medication intake through a smartphone app. The app reminds patients, confirms medication intake and sends this information to healthcare providers. 

This tech has shown success in improving adherence to treatment, particularly for conditions that require consistent medication.

‍

2.Tempus

Tempus is an American health tech company. It supports healthcare professionals in making a more informed decision. Tempus relies on machine learning to analyze genetic and clinical data, helping create more personalized treatments and precision medicine solutions for oncology, cardiology, and depression. 

3.HealthTap

HealthTap is a U.S.-based primary care platform that offers affordable services with or without insurance. They focus on making healthcare accessible through virtual consultations.

Natural Language Processing (NLP) powers HealthTap's virtual assistant, which helps guide patients through symptom checks and provides health information. This technology supports patient engagement and helps reduce the workload for healthcare providers.

AI + healthcare, a future worth building

AI holds the promise to transform healthcare by taking care of some of the industry's most challenging issues.

Beyond automating tasks or improving diagnosis accuracy, AI has the potential to help healthcare providers focus on what matters most—patient care and help with a better work-life balance.

The path forward requires strong partnerships between AI companies and healthcare organizations, with a shared focus on meaningful, patient-centred outcomes, as well as sustainable profitability. 

To make this vision a reality, it is extremely important that AI-assisted solutions are implemented responsibly, ensuring they are cost-effective and inclusive, leaving no one behind. 

We believe that with ongoing research, thoughtful development, and a human-centered approach, technology is not only going to get smarter but also become a valuable ally, transforming healthcare into a system that addresses current challenges, genuinely improves people’s lives, supports families, and uplifts the dedicated workforce behind it. 

This future is one where, with the help of AI, healthcare professionals can focus on what they do best—healing, connecting, and instilling hope.

Devs working with Drupal and Twig templates know the effort it takes to convert these into Single Directory Components (SDC). The process can often be manual and time-consuming, involving the creation of multiple configuration files. To streamline this, we’ve introduced a code-gen tool designed to simplify and speed up the conversion of your Twig components into SDC format.

SDC started as an experimental feature in Drupal, gaining traction with each release. This shift ties directly into the Experience Builder, which will rely on SDC as the standard for future page-building within Drupal. As this approach becomes essential for developers, our tool helps to get you future-ready, making it easier to transition and work easily with the upcoming changes.

Here’s a closer look at how the tool works and why it’s set to change the dev workflows.

What is Twig and SDC?

We don't really need to get into too much detail here, but it helps set the stage for the rest of the blog. So, Twig is a widely-used templating language in Drupal that makes it easier to manage the display of your website’s frontend. While Twig helps with organizing HTML templates, moving to Single Directory Components (SDC) introduces another layer of complexity. SDC requires separate configuration files, usually in YAML format, that define the properties and structure of each component.

Manually creating these files can be a hassle, especially for larger projects. That’s where the code-gen tool comes in.

Introducing the code-gen convertor 

This tool was created with a few key goals in mind—coding isn’t just a technical process, it’s a creative one. Devs worldwide are constantly working on tight deadlines, and we wanted to give our code friends a way to save time. Whether they’re looking to speed up the conversion of Twig templates into SDC or simply take a break to explore what’s latest in AI.

The Twig to SDC Converter simplifies the conversion of existing Twig templates into SDC-compatible components, saving time and reducing manual effort. By analyzing the provided Twig code, the tool automatically generates the .component.yml file, eliminating the need for manual setup entirely.

Here’s how it works in a few easy steps:

Upload Your component files

Simply zip your component folders, which may include Twig files, CSS, JavaScript, and other assets. You can upload multiple components at once, simplifying the process and saving time.

Automated processing

Once uploaded, the tool gets to work. It analyzes and processes each component, automatically generating a .component.yml file that includes all necessary properties, example values, and configurations for the component to work seamlessly within SDC.

Download the results

After the conversion is complete, you can download your components along with their newly generated .component.yml files. You also have the option to delete your uploaded files right away, and for added security, any remaining files will be automatically deleted after 24 hours.

Smart features

The tool uses built-in intelligence to:

• Identify component properties and types based on your Twig code.
• Generate example values that can be used within Experience Builder.
• Keep you informed with real-time progress logs as it processes your components.

For example, if you upload a Button component, the tool can detect props like variation and label, along with any default values.

Bringing it all together

While the tool streamlines the conversion process, it might not cover every scenario perfectly. Some props or specific cases may need a little extra attention, which is why we recommend reviewing the generated files to ensure everything fits your unique project requirements.

Think of our Twig to SDC convertor as a helpful assistant that takes care of the heavy lifting, while you fine-tune the details where needed.

Give the Twig to SDC Converter a try and see how it can streamline your workflow. We’d love to hear feedback and ideas from the community—whether it’s suggestions for this tool or thoughts on future automation tools you’d like to see!

Today's AI, powered by machine learning algorithms, has introduced us to semantic search. Essentially, semantic search interprets queries in a more human way by not just reading the keywords we type but also understanding the context, emotions, and intent behind our queries.

In our first blog in the semantic search series, we discussed how it can transform how we search. We also presented a demo application that we built to experiment with semantic search.

In this blog, we will explore a combination of Pinecone and OpenAI which are emerging as one of the key players in creating intelligent, user-friendly AI experiences. We'll also unravel the code that powers our demo application and illustrate how AI is revolutionizing data search.

Our approach to building the solution

We are moving away from traditional keyword-based searches towards searches that grasp the purpose and context behind our query. A key element enabling this smarter search is the integration of embedding models within Large Language Models (LLMs).

These models create vector embeddings that represent data in a multi-dimensional way, which helps in understanding content on a deeper level.

Tackling challenges with the right tools

While the potential of semantic search is impressive, there are some challenges in making it work effectively. These challenges include handling a robust infrastructure, reducing the time it takes to get search results, and keeping data up-to-date to ensure it remains relevant and useful.

However, when we have the right tools in place, these challenges become much easier to handle. An optimized vector database, for example, enhances the user experience by reducing delays and enabling real-time updates to the search results. It means we no longer have to choose between fast query responses and keeping our data current, resulting in a smoother and more efficient search experience.

How vector databases enhance semantic search

In semantic search, a vector database plays a crucial role as a storage hub for data embeddings. These embeddings capture the intricate contextual nuances of data.

When we perform a search query, instead of just matching words, we're looking out for vectors that carry similar meanings. This action not only sharpens the relevance of search results but also tailors them to fit the context. It proves beneficial in various scenarios, such as:

• Precise information retrieval: It helps teams to precisely locate specific internal data or knowledge without sorting through irrelevant information.
• User-friendly applications: It fine-tunes search results to match individual queries and elevates user engagement.
• Enhanced data integration: It amplifies search capabilities for large user bases by combining insights from diverse data sources.
• Personalizedrecommendation systems: Users receive suggestions closely aligned with their search history and preferences, making their experience more personal.
• Detecting anomalies in data streams: It assists in identifying and flagging outlier data or information that significantly deviates from established patterns, ensuring data quality and consistency.
• Streamlined content classification: It automates the grouping and labeling of data based on similarities, simplifying data management and utilization.

Why we chose Pinecone as our vector store

While looking for the right solution to seamlessly implement semantic search, we needed something that could easily work with our systems.

Pinecone stood out as a great choice, thanks to its user-friendly REST API. Apart from being easy to integrate, Pinecone provides:

• Extremely fast search results, even when dealing with huge amounts of data.
• Ability to update our existing data points in real-time, ensuring that we always have the latest information at our fingertips.
• A fully managed platform, so we can focus on using it rather than dealing with maintenance.
• Flexibility in terms of hosting options, including platforms like GCP, Azure, and AWS.

Selecting OpenAI as our embedding service

To enhance the efficiency and precision of our semantic search capabilities, selecting the right embedding service was important.

We found OpenAI's Embedding API to be an ideal choice for achieving unmatched contextual understanding in data processing.

Here's why we opted for OpenAI:

• Deep text understanding: OpenAI's embedding service is known for its ability to understand text deeply, making it skilled at finding important patterns and connections in large datasets (powered by the advanced GPT-3 model).
• Easy integration: OpenAI's well-documented API seamlessly fits into our existing systems, making it simple to add advanced search features. It's a quick choice for trying out new ideas.
• Continuous improvement: OpenAI is committed to making its services better over time. It means we can expect regular updates and enhancements, ensuring our search capabilities stay at the cutting edge of technology.

Opting for Rust as our backend API

To enhance our backend infrastructure, selecting the right programming language is crucial, but not necessarily restrictive.

Rust stood out as a great option, but it's not a strict requirement. Languages like JavaScript, Python, or any language capable of making cURL requests and handling data can work just fine.

However, there were some compelling reasons that made us choose Rust, especially as we explore Rust-based libraries for near real-time LLM inference on cost-effective hardware, which we'll discuss in a future instalment of this series.

Here's why Rust was an excellent choice for our API development:

• Speed: Rust is known for compiling efficient machine code, delivering performance similar to languages like C and C++. This speed makes it a strong choice for high-throughput APIs, although it's important to mention that our current reliance on a third-party API presents a challenge in achieving response times under 150 ms.
• Concurrency: Rust's ability to prevent data conflicts in concurrent programming with its skill in managing asynchronous tasks is very useful when handling many API requests happening simultaneously.
• Memory Safety: Rust proactively detects common errors like trying to access non-existent data, reducing crashes and security vulnerabilities during the coding process. It also works seamlessly with tools like the rust-analyzer plugin in VS Code, making debugging and development smoother.
• Compatibility: Rust works well with C libraries, making it easy to directly use functions from these libraries. This provides flexibility when integrating with existing systems.
• Community and Ecosystem: Rust benefits from a growing library collection and an active community. It's becoming a central hub for strong tools needed to create web services and APIs. The fact that enthusiasts have creatively made LLMs work with Rust, highlights it as a quick and capable option for more experiments.

Streamlining data and AI embeddings on Pinecone

In this section, we'll walk you through the process of collecting data, generating AI embeddings using the OpenAI Embedding API, and conducting semantic search experiments within the Pinecone Vector Database.

OpenAI models, like GPT-3, have been trained on a vast and diverse collection of text data. They excel at capturing complex language patterns and understanding context. These models transform each word or phrase into a high-dimensional vector. This process, known as embedding, captures the meaning of the input in a way that's easy for systems to understand.

For example, the word "lawyer" might be represented as a 1536-dimensional vector (using the 2nd gen OpenAI embedding API model text-embedding-ada-002, which is based on GPT-3). Each dimension in this vector captures a different aspect of the word's meaning.

The below example is from Tensorflow. It uses a completely different model, but the concept remains the same.

These embeddings play a crucial role in semantic search. When a user enters a search query, the AI model creates an embedding of that query.

This embedding is then sent to the vector database, such as Pinecone in our case, which finds and retrieves the most similar vectors, essentially providing the most contextually relevant results.

Think of it as translating human language into a format that machines can easily grasp and work with effectively. By generating these embeddings, OpenAI models enable us to achieve more precise and context-aware search results, a significant advancement over traditional keyword-based search methods.

Let's take an example: imagine a user searching for "rights of a tenant in Illinois." With a traditional keyword-based search, you'd get documents containing those exact words. But when we use an AI model to create embeddings, it understands the real meaning – that the user is looking for information about tenant rights in Illinois.

The system then fetches relevant results, even if they don't use the exact phrasing of the query but discuss the same idea. This could mean providing a comprehensive guide to tenant rights, mentioning a relevant court case in Illinois, or sharing a related law statute. In the end, it gives the user a more detailed and helpful response.

It's the combination of OpenAI's Embedding API and Pinecone's efficient vector search that makes this enhanced, contextually-aware search experience possible.

Implementation of our solution

Please note that this experiment was tailored for a specific website.

Here's what we did:

Step 1

Data Cleanup and Preparation: Our first step involved cleaning up the data and making sure it was compatible with Pinecone. We used Python for data preparation since it is simple for handling large datasets.

Data Collection: We collected a CSV file with over 1600 rows, all related to legal assistance from the Illinois Legal Aid Online site.

Pinecone Database Requirements: Pinecone needs data to have three specific columns: id, vectors, and metadata. We ensured our data met these requirements for seamless integration.

Before we proceed to create the vector, let's organize the data. Column names have been shortened for data privacy.

Note: We are not using metadata in our current experiment. It's primarily used for filtering and faster querying, or as a means to transmit data while querying in a different environment. In our case, Pinecone Query API will perform more efficiently without the inclusion of metadata.‍

# Setup pinecone structured data
from slugify import slugify

def create_doc_index(row):
    """
    An apply function to alter the content.
    
    Creates a new column called "id".
    Which contains the slugified title.
		
		Eg. "hello world" will be 'hello-world'
    """
    row['id'] = slugify(row['Title'])
    return row

def create_data(row):
    """
    An apply function to alter the content.
    
    Creates a new column called "data".
    Which contains the following as a single text string:
    '
        Title: {Title column value},
        Description: {Content description column value},
				...
				Content: {Content column value}
    '
    """
    row['data'] = f"Title: {row['Title']}, Description: {row['Content description']}, Content: {row['Content block']}"
    return row

def create_metadata(row):
    """
    An apply function to alter the content.
    
    Creates a new column called "metadata".
    Which contains the following as a dict:
    {
        'Title': {Title column value},
        ...
        'Legal category (select all that apply)': {Legal category (select all that apply) column value},
    }
    """
    row['metadata'] = {
        'Title': str(row['Title'] if row['Title'] else ''),
        ...
        'Legal category (select all that apply)': str(row['Legal category (select all that apply)'] if row['Legal category (select all that apply)'] else ''),
    }
    return row

Here's an overview of what the data looks like after we completed the initial cleanup and processing. We applied the functions we created earlier to each row in the database.

Next up is the generation of AI embeddings for the vector database. Please note that you'll need OpenAI API keys for this step.

def embeddings(text):
    # Make API request to OpenAI's Embedding endpoint
    response = openai.Embedding.create(
        model="text-embedding-ada-002",  # The model to use for the AI embeddings
        input=text,  # The text to embed
    )
    # Retrieve and return the AI embedding from the response.
    return response['data'][0]['embedding']

# Generate the first 10 components of the AI embedding for "Hello"
embeddings("Hello")[:10]

‍
For example, when we input the test string "hello," we receive the following set of embeddings as output.

[-0.021849708631634712,
 -0.007138177752494812,
 -0.028344865888357162,
 -0.02456468529999256,
 -0.023603402078151703,
 0.028864478692412376,
 -0.012243371456861496,
 -0.0028562454972416162,
 -0.00829431600868702,
 -0.00539747579023242]

‍
We apply this AI embedding function to all the rows in our dataset. You can see the vector column in the image below.

Now, on to the final step - uploading this data to Pinecone. We are using the gRPC protocol provided by the Pinecone library, which makes this upload faster, taking less than 15-30 seconds in total.

# Creating the Pinecone index only if it does not exist
if index_name not in pinecone.list_indexes():
    pinecone.create_index(
        name=index_name,
        dimension=1536,  # specifying the dimension for the index vectors
        metric='cosine'  # specifying the metric for vector comparison
    )

# Creating an interface to interact with the Pinecone index
index = pinecone.GRPCIndex("hybrid-legal-doc-search")

# Inserting/updating vectors in the index from our DataFrame 'ddf'
index.upsert_from_dataframe(ddf, batch_size=100)

‍
Take a peek at the index statistics using

index.describe_index_stats()

{'dimension': 1536,
 'index_fullness': 0.0,
 'namespaces': {'': {'vector_count': 1697}},
 'total_vector_count': 1697}

‍

Lastly, when querying in Pinecone, we need to provide the AI embeddings to get the relevant results.

You might wonder why we're generating AI embeddings before uploading data to Pinecone and prior to querying. The reason is that embeddings can be created using various models, and they are not always compatible with one another.

Different models like Bert and simple Word2Vec, among others, can be used to perform the same task, and they produce embeddings that may not work interchangeably.

This is why it's important to have the embeddings prepared in advance to ensure a smooth and consistent search experience.

# Querying the Pinecone index with the query vector
# 'top_k' indicates we want the top 3 results and 'include_metadata' set to False indicates we don't want additional metadata
xc = index.query(embeddings("family and legal advice"), top_k=3, include_metadata=False)

print(xc)

# Output:
{'matches': [{'id': 'someone-called-dcfs',
              'score': 0.8318765,
              'sparse_values': {'indices': [], 'values': []},
              'values': []},
             {'id': 'divorce-and-parental-responsibilities',
              'score': 0.81308925,
              'sparse_values': {'indices': [], 'values': []},
              'values': []},
             {'id': '7-things-i-can-do-to-safely-talk-to-my-lawyer',
              'score': 0.81211287,
              'sparse_values': {'indices': [], 'values': []},
              'values': []}]

‍
Step 2

Next, we'll create an API that can manage both AI embedding and querying processes seamlessly.

Here is an overview of the backend development process.

Here is the code for Rust backend that couples our Pinecone and OpenAI APIs together.

// The "q" route handeler
pub async fn query_q_api(
    // The query
    query: HashMap<String, String>,
	// Web client
    client: Arc<Client>,
	// LRU Cache
    q_cache: Arc<Mutex<LruCache<String, Vec<ResponseMatch>>>>,
) -> Result<impl warp::Reply, warp::Rejection> {

    // Extracting and decoding the query parameter 'q'
    let input = decode(query.get("q").unwrap_or(&String::new())).unwrap();
    let q_cache_key = input.clone();

    // Attempting to get a response from the cache if it exist
    {
        let mut q_cache = q_cache.lock().await;
        if let Some(cached_response) = q_cache.get(&q_cache_key) {
            // If a response is found in the cache return the cached response
            return Ok(warp::reply::json(&cached_response.clone()));
        }
    }

    // If no response was found in the cache, fetch new data
    let response_matches = fetch_new_data(client, input).await;

    // Put the new data into the cache for future use
    {
        let mut q_cache = q_cache.lock().await;
        q_cache.put(q_cache_key, response_matches.clone())
    }
    
    // return the fetched data as the response
    Ok(warp::reply::json(&response_matches))
}

‍
For our demo, we've implemented a short-term, in-memory cache to prevent unnecessary API calls.

However, in the future, we aim to introduce an on-disk cache and implement local semantic search capabilities for queries with similar meanings. It will enhance the efficiency and responsiveness of our system down the line.

// Creating a shared LRU cache for "q" route and defining the route
    let q_cache = Arc::new(Mutex::new(LruCache::<String, Vec<ResponseMatch>>::new(CACHE_SIZE)));
    let q_route = warp::path("q")
        .and(warp::query::<HashMap<String, String>>())
        .and(with_client(client.clone()))
        .and(with_q_cache(q_cache.clone()))
        .and_then(query_q_api);

‍
Next, we have a helper function ‘fetch_new_data’. This function handles calls to both the OpenAI and Pinecone APIs, ensuring a smooth flow of data retrieval and processing.

// This function fetches new data from OpenAI and Pinecone APIs
pub async fn fetch_new_data(client: Arc<Client>, input: String) -> Vec<ResponseMatch> 
    
    // Getting the AI embeddings from OpenAI API
    let openai_resp = get_openai_response(&client, input, &openai_org_id, &openai_api_key).await;
    let embedding = &openai_resp.data.get(0).unwrap().embedding;

    // Getting the top matches from Pinecone API
    let pinecone_resp = get_pinecone_response(&client, embedding.to_vec(), &pinecone_api_key, &pinecone_endpoint).await;
    
    // Converting the Pinecone response to a list of ResponseMatch and returning it
    return pinecone_resp.matches.into_iter().map(|match_| {
        let metadata = match_.metadata.unwrap_or(json!({}));
        ResponseMatch {
            id: match_.id,
            score: match_.score,
            title: metadata.get("Title").unwrap_or(&json!("")).as_str().unwrap_or("").to_string(),
            description: metadata.get("Description").unwrap_or(&json!("")).as_str().unwrap_or("").to_string(),
            url: metadata.get("url").unwrap_or(&json!("")).as_str().unwrap_or("").to_string(),
        }
    }).collect::<Vec<ResponseMatch>>();
}

// This function sends a POST request to the OpenAI API to get the AI embeddings for the input text
pub async fn get_openai_response(client: &Client, input: String, openai_api_key: &str, openai_org_id: &str) -> OpenAIResponse {
    
    // Sending the request to OpenAI API and getting the response
    let response = client
        .post("https://api.openai.com/v1/embeddings")
        .header("Content-Type", "application/json")
        .header("Authorization", format!("Bearer {}", openai_api_key))
        .header("OpenAI-Organization", format!("{}", openai_org_id))
        .json(&json!({
            "input": input,
            "model": "text-embedding-ada-002"
        }))
        .send()
        .await.unwrap()
        .json()
        .await.unwrap();
    
    response
}

// This function sends a POST request to the Pinecone API to get the top matches for the input embedding
pub async fn get_pinecone_response(client: &Client, embedding: Vec<f64>, pinecone_api_key: &str, pinecone_endpoint: &str) -> PineconeResponse {
    
    // Sending the request to Pinecone API and getting the response
    let response = client
        .post(pinecone_endpoint)
        .header("Api-Key", pinecone_api_key)
        .header("Content-Type", "application/json")
        .json(&json!({
            "vector": embedding,
            "topK": 3,
            "includeValues": false,
            "includeMetadata": true
        }))
        .send()
        .await.unwrap()
        .json()
        .await.unwrap();
    
    response
}

‍

Currently, we're experiencing significant delays with both the Pinecone and OpenAI APIs. Our goal is to cut down this delay further by exploring on-premises solutions.

It involves considering options like Bert for AI embedding and Milvus or similar open-source vector databases that can be used locally.

With the API components in place, take a look at what the demo frontend has to offer.

Here's a simple "useEffect" method within our React app. It's responsible for updating the search results in real-time as the user types in their query.

// searchText is the text from the search box.
useEffect(() => {
		// Check in the client-side cache if found set the results
    if (searchCache.current[searchText]) {
      setResults(searchCache.current[searchText]);
    } else {
			// When the user enters more than 2 letters the fetching process starts
      if (searchText.length > 2) {
				// Defining the fetch function
        const fetchData = async () => {
          try {
						// Calling our previously developed Rust API
            const response = await fetch(
              process.env.REACT_APP_BASE_API_URL + `/q?q=${encodeURIComponent(searchText)}`
            );
            const data = await response.json();
						// Store to user cache, just in case the user searches for the same Query
            searchCache.current[searchText] = data;
						// Set results
            setResults(data);
          } catch (error) {
            console.error("Error fetching data:", error);
          }
        };
				// Caling the fetch function
        fetchData();
      }
    }
	// monitoring the searchText var for changes
  }, [searchText]);

‍
What’s next

We tried the RAG approach which involved feeding the results into a ChatGPT-like system, ultimately providing users with more personalized results without the need to navigate through numerous blog pages. We employed interesting strategies and encountered some challenges with this approach. You can delve deeper into our journey by checking out our next blog in this series.

We are soon planning to launch a platform where you can play with all these experiments. As we move forward, our focus is on matching the right technology with the needs of our customers. The field of semantic search and vector similarity is full of exciting possibilities, such as creating recommendation engines based on similarity.

Right now, we're actively working on several Proof of Concepts (PoCs), where we're balancing speed and accuracy depending on the specific application. We're exploring various models, including BERT-based ones and more, beyond the standard 'text-embedding-ada-002.'

We're also attentive to our customers' preferences for platforms like Azure or GCP. To meet these preferences, we're adjusting our approach to include models recommended by these providers, aiming to create a versatile system that can effectively serve different use cases, budgets, and unique requirements.

‍

Artificial Intelligence (AI) has transformed the way we search for information. Search behavior has evolved from simple keyword searches like "running shoes" to specific and personalized queries like "comfortable shoes for jogging” or “athletic shoes for beginners”. This change will continue as customer behavior evolves with technology.

A McKinsey study reports that 71% of users expect personalized search results and are often frustrated when these expectations are not met.

The traditional keyword-based systems do struggle with natural queries and miss the context in the process, while semantic search understands the intent behind the query and delivers more relevant results.

Hence, we leveraged AI by merging our understanding of Large Language Models (LLM), vector databases, and Retrieval Augmented Generation (RAG), to create an advanced semantic search system to respond to complex but natural human queries.

Limitations of traditional searches and LLMs

We examined the problem closely to ensure that our solution effectively addresses the challenges with traditional keyword-based systems and to build an outcome, that can be put to use.

Here are some reasons why we chose to go ahead with our semantic search and custom result generation experiment:

Challenges with traditional search systems

1. Speed and efficiency: Traditional search methods may not deliver quick or efficient results whereas semantic search can sort through large amounts of data in real time to find results faster.

2. Relevance: Traditional search relies on exact keyword matches and often misses the context of the search queries. Semantic search, on the other hand, understands the intent behind a query and looks for relevant results.

3. Complex infrastructure: When it comes to traditional searches, integrating AI functionalities can be quite a complex task. It involves setting up additional databases and fine-tuning models just to understand search queries better.

4. High latency: Dealing with large sets of records in traditional databases becomes a slow process and at times expensive when it comes to storing and searching for information.

Drawbacks of foundational LLMs

LLMs are advanced AI-driven models designed to understand and generate human-like text. Some examples of foundational LLMs include OpenAI's ChatGPT or GPT-3.5-Turbo, and Anthropic's Claude.

1. Static nature: LLMs, such as GPT, have a fixed knowledge base, and they can't adapt to or include new real-time information once they've completed their training.

2. Lack of domain-specific knowledge: LLMs are trained for general tasks and would not have specific details from private or recent datasets, like a company's latest products or updates.

3. Black box functionality: It can be quite challenging to understand where the answers generated by an LLM come from as it's not always clear which sources or reasoning it relies on.

4. Cost and inefficiency: Developing and launching foundational models like GPT demands significant resources and makes it difficult for many organizations to customize them for specific tasks.

5. Limited contextual understanding: Although LLMs can produce clear responses without extra assistance (like using RAG), they may not consistently provide the most context-aware answers. For example, helping a customer book a flight often requires real-time data and context, which a basic GPT model might lack.

6. Accuracy: Using LLMs without any tweaks or adjustments can sometimes result in errors, incorrect information, or less-than-ideal answers for specific, detailed tasks.

User queries have changed from simple keyword-based questions to more complex, context-heavy inquiries. This shift shows how people increasingly depend on technology to understand detailed and varied questions.

While traditional keyword-based systems such as Solr, Google PSE, and Algolia expect users to adjust their questions to fit the system's limitations, newer AI-enhanced platforms are setting a new standard. They're now adapting to users, grasping their intentions, understanding the context, and even picking up on emotions.

This change marks a move towards more intuitive, conversational, and human-friendly interactions with technology, reflecting our natural desire for clear, relevant, and instant responses.

Designing a semantic search and RAG system

Our goal was to create a system that could understand language more effectively, efficiently manage vast amounts of data, and deliver personalized results. We focused on addressing the challenges posed by traditional search methods and the drawbacks of foundational LLMs.

We aimed to use our insights to build a system capable of finding the most relevant information and providing useful answers. Our approach was simple, focused on practical solutions, and leveraged the following key features:

AI-powered search

We wanted to develop a search system that uses semantic search. We achieved this by taking user search queries and converting them into numerical vectors using machine-learned meaning in the backend. These sets of numbers or vectors were then matched against a database of similar vectors to identify the most similar results.

We enhanced the search results by understanding the user's intended meaning and providing more relevant results. It's worth noting that semantic search can sometimes produce inaccurate results when dealing with short queries, typically comprising just two or three specific keywords.

Contextual response generation with AI

We improved the user experience by providing responses that are aware of the context of the user's queries through an 'Ask AI' function. This function considers all the relevant results gathered from previous smart searches.

The data returned from the smart search can be used as context for the user's query and sent to the OpenAI Chat API to generate a continuous text response. This process is known as Retrieval Augmented Generation.

With these features, we believe our AI-driven search system can greatly enhance the way people search for information.

Our toolbox

We chose the following tools for storing data:

• Pinecone: It's an AI solution that uses vectors for scalability and doesn't require any maintenance or troubleshooting. It speeds up data searches within milliseconds, applies filters to metadata, and supports indexes that help find precise search results for various tasks.
• Meilisearch: This is a free and open-source alternative to Algolia, that comes with an exciting experimental feature allowing us to store vectors with traditional data. It combines the key features of both Algolia and Pinecone and delivers results in just milliseconds.
• OpenAI API: This API gives access to all the general-purpose models, including GPT-4 and more. It can be easily integrated into applications to handle complex tasks and enhance experiences with AI.

As for the demo, we chose the Tokio web server (built on Rust) for the backend and its main query handler due to its high performance. It's reliable and provides a safe environment, making it an ideal choice for building an API.

Combining AI with Rust brings benefits like speed, handling multiple tasks simultaneously, ensuring memory safety, and the ability to work with C libraries. Additionally, Rust provides a supportive community and a growing set of libraries for creating web services and APIs.

The demo search application

Challenges with the RAG system

Limited model context windows and large text blogs

A context window is like the number of tokens or text the model can take in before it generates more text. For example, common models like GPT-3.5-Turbo have a context window of 4,096 tokens, which equals about 3,000 English words. Meanwhile, bigger models like GPT-4 have 32,768 tokens, which would be about 24,500 words. (These are approximate figures and can vary depending on the text and how it's processed by the tokenizer model.)

When we add more data, these can become limitations. Some models, like Claude 2, have a wider context window of 100,000 tokens, but you start seeing diminishing results, and issues like hallucinations pop up.

We tried out different approaches, but for our experiment, we went with the simplest one, "trimming." However, we considered the Pinecone scores before cutting out parts of the context.

We're also exploring other ideas, like creating smaller vectors and using their metadata to connect back to the main data. This could potentially save a lot of tokens.

A steerable prompt for the GPT model

We tried out various prompts to arrive at what we have now. In this experiment, we used the following prompt: (Note that prompting changes with models and version. The one we used here is GPT-4 2023 June 13th snapshot)

    
    {
    // The 'system' role is what drives the model, in other words, defines the goal or purpose of the model to behave.
        role: "system",
        content: `You are an advanced legal aid search engine bot, developed by ILAO - Illinois Legal Aid Online. Your primary role is to deliver highly relevant, accurate, and useful search results to users based on their Query and the available Context.
    Please follow these guidelines strictly:
    1. Provide responses directly related to the user's Query. If the query is unclear or insufficient, summarize the Context and include any pertinent details about the Query.
    2. Don't ask the user questions as they don't have the capability to respond.
    3. Don't introduce yourself. The goal is to provide search results swiftly and efficiently.
    4. Strive to provide the best possible results for each Query, like a dedicated legal search engine. 
    5. Use the Context provided to craft comprehensive, succinct, and user-friendly answers to the Query.
    6. Refer to results from the Context using [context-id] notation for citation. For example: 'some text [1] some other text [2]'.
    7. Do not include the full text of cited sources. These will be managed by separate software. Try to avoid citing the sources too many times.
    8. In cases where the Query relates to multiple subjects sharing the same name, formulate separate responses for each subject to ensure clarity.
    9. Utilize markdown formatting for clarity and readability.
    10. Limit responses to a maximum of 300 words to provide concise and focused answers.
    Remember, your ultimate goal is to assist users in navigating legal information quickly and accurately, in line with the mission of Illinois Legal Aid Online.`,
    }
    
    
    

This prompt may evolve as we work on improving it. We're looking to refine it by referring to the details provided in "What are tokens and how to count them?" by OpenAI.

Streaming responses in JS

When we were working on the demo, the official OpenAI JS library didn't have support for streaming responses. So, we used an alternative library called "openai-ext," which not only allowed us to implement streaming responses but also made it easier to manage the buttons' states.

API latency

Pinecone claims that some customers can get their search results in under 100 ms, but we haven't been able to achieve the same speed with our HTTP requests to the API. We're still figuring out how to reach that kind of latency.

One way could be by using gRPC implementation in Python, or maybe some other method we haven't tried yet. We're also exploring options for on-premises solutions with custom search algorithms that might give us response times faster than 100 ms.

Lately, we've noticed that the OpenAI embedding API has been slowing down. Initially, a few results occasionally took over 300 ms, but now it's happening more often. It's not as fast as we'd like for an instant search experience.

To make it work smoothly, either OpenAI needs to upgrade its servers to generate embeddings faster, or we'll need to find on-premises solutions, like using local Bert models for embeddings, which could give us an average response time of less than 60 ms.

Dive deeper into our demo search experience

Take a closer look at our demo search experience in action. You can explore our semantic search in action in two informative blogs:

Revolutionizing Search with AI: Diving Deep into Semantic Search - This blog will give you an inside look at our demo application, explaining how we implemented semantic search and built the infrastructure using Rust.

Revolutionizing Search with AI: RAG for Contextual Response - In this blog, we uncover the inner workings of RAG paired with GPT. You'll discover how we transform user queries into personalized responses that make interactions feel truly human.

What's next in AI search series

We are continuously seeking new ways to make the experience more personal for our clients and their users. Along with tackling current challenges, here are some of the ideas we're exploring:

On-premises and open-source alternatives

Our clients have diverse needs. Some prefer on-premises solutions, while others rely on open-source software. Non-profit organizations often seek a balance between the two. To cater to this variety, we're considering different technologies to expand our offerings. These include cloud-based LLMs like Azure-hosted GPTs and Claude 2, open-source LLaMA 2, vector solutions such as local Milivus, and hybrid search solutions like Typesense and Meilisearch.

Fine-tuning search experience

We're running experiments with feedback loops and using user data to personalize search results and improve the one-on-one user experience. OpenAI recently announced that their GPT-3.5-turbo model can be fine-tuned with custom data, which makes the Reinforcement Learning through Human Feedback (RLHF) approach easier and more cost-effective.

Summary

We believe that semantic search can transform the search experience, making it more intuitive and user-friendly. It has the potential to simplify the process of finding information, even when dealing with complex queries.

Our journey with AI continues to evolve through these ideas and search experiments. We are constantly striving to innovate solutions that can bring exceptional experiences to today's digital platforms, setting the stage for tomorrow's personalized AI-driven Digital Experience Platforms (DXPs).

We are living in the age of data and to thrive in this era we need to turn this data into knowledge. Data in an organization is scattered over lots of different mediums, whether in an email, dropbox, or on your Slack. In an attempt to create value from our data, we experimented with QED42’s Slack public data and transformed it into an intelligent model using Machine Learning. 

QED42 and Slack

QED42 relies on Slack for the day to day interactions between our employees. Since remote work/flexible work timings have always been a part of our work culture, we introduced an ‘Update Message’ channel on Slack. The purpose of this public Slack channel is to help employees keep a track of their teammates' availability. 

Here are some examples of the update statuses added regularly by our teams:

• Whether he/she is working from home (WFH)
• Taking a short break (AFK)
• Leaving for the day (LFD) 

Our Project

This public Slack channel carries an enormous pool of data coming from multiple users that span over months or even a year! And so, we thought of experimenting with the Slack data at hand! The purpose of applying Machine Learning on our Slack public channel was to offer a better way to listen, learn and engage with the team. 

The model was built with an intent to understand how things are evolving with us working 100% remotely with distributed teams and taking a data-driven approach to optimizing our communication. Although this approach was taken towards an experimental project, the same model has also seen its use in various other classification or predictions like ham or spam messages, sentimental analysis of the messages, etc. When it comes to feature extraction we have tagged the data with time. This method can be used to represent various time-series data like climate analysis, appointment no-shows, etc.

Our Objective

The sole objective of this project was to visualize how employees communicate to identify patterns and optimize communication. We made sure that this implementation is 100% non-intrusive (we did not monitor private channels, groups, and direct messages). 

Our objective was to create a dashboard that displays the status updates of any employee for a given date. Moving forward, we also wanted to go beyond just fetching and displaying data from Slack, we aspired to predict a person’s status! Now, isn’t that interesting? 

Before we jump to the prediction mechanism, this is how we calibrated the data for our learning algorithm. 

Sniffing Data from Slack  

To launch our experiment, we had to retrieve the update channel messages from our Slack channel so that we could categorize the data into different buckets like AFK, WFH, LFD, etc. This can be achieved using any of the following methods:

Method 1: The CRON Job

This code snippet will regularly ask Slack to provide all messages in its history between a fixed timeline provided by the user. For this, we need to have an auth token (this token can belong to a Bot or a user). This code is implemented in Python 3.0 and we need the Slacker package for Python to perform this request. 

while(True):
  response = slack.channels.history(
    channel = 'C91H4BC0Z',
    latest = lastTimestamp, # this can be zero or the timestamp of the last data
    oldest = time,
    count   = pageSize
  ).body
  messages.extend(response['messages'])
  if (response['has_more'] == True ):
    lastTimestamp = messages[-1]['ts'] # -1 means last element in a list
  else:
    break

However, there is one drawback with this method – we need to set this as a Cron job. An alternative to this is that Slack allows us to perform this function on a real-time basis which can be achieved using method 2.

Method 2: Real-Time Monitoring Messages (Event handler)

Fig 2.1

Building an event handler is an apt way of handling this situation, but the configuration might take longer than our first method.

This method will help you interact with the end-user in real-time. For this, you need to create a Slack app and assign it to the team. Also, you need to add a bot user (you can find it in Fig 2.1 red box) for the app. The main purpose of this bot is to interact with people since we are categorising the messages into specified buckets. It is common for users to add random messages on this channel and the purpose of the bot is to discourage people from doing so. 

Something like this:  

Fig 2.2

Enabling Event subscription:

• To enable event subscription, you need an HTTPS URL for the post (basically the Slack will hit this URL if an event has occurred). 
• If you are using it for development purposes you need to run a tunnel-like ngrok proxy tool. Get the base URL using the tool, append ‘/slack/event’ and paste on the field (field green box in fig 2.1). 
• Go to OAuth and Permission tab(black box fig 2.2) get the Bot User OAuth Access Token. 
• After that you need a Signing Secret for authentication, you can find this inside the Basic information session (blue box inside fig 2.1) Also, don’t forget to install the app in the workspace. 

from slackeventsapi import SlackEventAdapter
from slack import WebClient


slack_signing_secret = "SLACK_SIGNING_SECRET"
slack_events_adapter = SlackEventAdapter(slack_signing_secret, "/slack/events")

slack_bot_token = "SLACK_BOT_TOKEN"
slack_client = WebClient(slack_bot_token)

@slack_events_adapter.on("message")
def handle_message(event_data):
  message = event_data["event"]
  keys_list = message.keys()
  keys_list = list(message.keys())
  avoid = ["subtype","thread_ts"] #to avoid reply and emoji's
  if message["channel"]=='C010YHT8DM2':
      if not any(elem in keys_list  for elem in avoid):

           message.get('text')
          message["user"]

slack_events_adapter.start(port=4000)

Categorizing messages into buckets

Once the messages are collected, the next step is to categorise them into different states like WFH, AFK, LFD. The only way to achieve it is by going through the words, here is where the Natural Language Process (NLP) comes into play. 

Once we get the text we tokenize the text and search for the keywords. For example, if an employee wants to avail work from home he/she might send the message like ‘WFH’ or ‘work from home’ similar pattern implies for the other buckets too. The idea is to distinguish the messages using these keywords.

template={
   'WFH' :[['wfh'], ['work','home']],
   'AFK' : [['afk'], ['away', 'keyboard']],
   'LFD' : [['lfd'], ['leaving','day', 'closing']],
   }
   #here status[0] will be WFH and status[1]=AFK...
   status = [0,0,0]
   message = message.lower() #messages is converted into lower case for the easier comparison
   message = message.replace('.','')
   tokens = word_tokenize(message)
   for element in template:
       #after this loop which of these conditions matches the conditions.
       for ele in template[element]:
           count = len(set(ele)&set(tokens)) #set gives list of matches in the both array
           if element == 'WFH':
             status[0] = status[0] + count
           elif element == 'AFK':
             status[1] = status[1] + count
           elif element == 'LFD':
             status[2] = status[2] + count
   state = status.index(max(status))
   if(status[state] != 0):
       status = switcher.get(state, lambda: "Invalid data") #running the switch case
   else:
print("invalid data")

PS: Although in the above sample, it made sense to use some specific keywords for classification, one can resort to NLP & Artificial Intelligence for classification as well. A couple of standard ways in which it can be done for more complex scenarios where we need to extract the intent from a statement are:

1. Document similarity:  This NLP based technique finds out the similarity between two documents by converting them into keyword word-count vectors and computing the distance between them using the cosine similarity function.
2. AI-based classification: Pre-trained ML/AI models are used to classify the messages into buckets. 

Predicting the state

Assigning a state to these messages is pretty straightforward, but the fun lies in taking things to the next level!

Availing work from home or taking a short break, completely depends on a particular person and his/her mood, but what if there is a pattern to it? 

For example, let’s say I am a person who takes my lunch break around 2 PM almost every day. Voila! There lies the pattern (I will be AFK at 2 PM). There could be similar patterns visible in other employees too.

Modelling the data 

The data collected from Slack cannot be fed directly to any learning algorithm. We need to first understand the data itself. Let’s consider a couple of scenarios: 

• In the case of work from home, it is the date that we should be concerned about
• For AFK and leaving for the day, it will be both the time and date 
• While constructing the datasets for WFH, we should also keep in mind the WFO (work from the office). If a person is not working from home on a particular date the entry for the data should be WFO the same applies to AFK. 
• For date, we need to consider a lot of specifications like a date with respect to the year, date with respect to the month, etc. 

Since there is no interdependence among the employees we should build different models for each person.

Algorithm

The final step was to wrap this data inside an algorithm. For that we have to go back to the data, with an initial look one can easily conclude that this data is inconsistent. This is obvious, considering that a person would mostly work for 100 days while he/she might avail work from home only for 10 days. 

Another factor that must be considered is overfitting. Most trivial machine learning algorithms like linear regression may be easy to understand and implement but are prone to overfitting. Gladly there are a ton of algorithms available to prevent overfitting. One of them is LASSO (Least Absolute Shrinkage and Selection Operator) regression. 

With the help of the LASSO regression, we can reduce overfitting by shrinking large regression coefficients similar to Ridge regression. It also has the added benefit of variable selection i.e. removing the unimportant variable by keeping their coefficient zero. This, in turn, removes the corresponding feature from the model. 

A tuning parameter named λ is used to control the strength of the penalty, resulting in the number of variables selected and their contribution to the model. 

As the value of λ increases few coefficients are selected. Unfortunately, this process failed in our scenario, giving a negative R2 score.  

This means that our system performed worse than a model which predicts the result based on the probability determined directly by the number of WHFs and WFOs in the data. But, why? One possible reason could be that the data is not linearly separable. Another reason could be the imbalance, which might have led to the model eventually ignoring most of the least represented class.

Before going further let us plot the data, here we are going to plot the field from the dataset using matplotlib. The graph will look like this:

Here the blue dots signify WFH and the yellow dots are WFO. The basic idea of Lasso or Ridge regression is to find a line that can be drawn across this graph and can separate the classes. But look how scattered the points are, it is difficult to separate with a single line. 

This problem urged us to think about other algorithms and we came up with the most common and yet one of the strongest algorithms of all - the Random forest. Random forest or Random decision forest processes its learning by building multiple decision trees. Decision trees are good for working with non linearly separable data. But, we can’t rely on a single decision tree to do this job because it is highly prone to overfitting. 

Random Forest will construct different decision trees and train them with a few of the samples from our dataset. The outcome is determined by tallying the individual outcomes or probability determined by the decision trees. Here too we faced the same demon which is the imbalance in the data.

For example, the ratio between WFH and WFO (Work from the office) might be 1:20, which weakens our model for detecting the less represented class (in this case WFH). We can build a workaround by keeping the threshold for classifying work from the office as high.

(LHS the probability of the WFH and RHS the probability of the WFO, you can in the last the probability for the WFO is less than the threshold and thereby the WFH will be triggered)

Obtaining the threshold

With a glance at the above image, one can understand that the probability of WFO is always higher than the WFH. The cause of this problem is because the samples of WFO are much higher than the WFH. Our next step was to determine the threshold value used to determine whether the given probability should be considered as WFH or WFO.  We had a few tricks under our sleeves to determine the threshold value. One was to create a formula that would give us the threshold point and the other one was to train a different ML model using the probability data. 

The formula that we developed computes the average WFH probability from using both WFH and WFO probabilities. The value thus computed is used as the threshold. To check which one will be better, we had to try them all and consider the one with more accuracy. We executed these methods with a selected group of people (with different WFH numbers) and built an analysis sheet out of it.

Even though the formula proved to yield promising results in terms of accuracy when it came to the statistics of True positive and True negative it turned out to be least effective. This forced us to put the data again through the ML algorithm (Random Forest). 

High-Level working of the model

Conclusion

Machine learning algorithms find patterns in data that humans cannot. The purpose of this application was to wrap our head around the ideology of creating automated solutions with Machine learning models and generate valuable statistics with the data in hand. 

Coming back to the application, we can’t conclude without exposing it to the real-time data even though the application gave a minimum accuracy of 70%. We planned to test out this application with our employee for around 6 months but things panned out differently due to the current pandemic situation. Currently, our application only evaluates WFH prediction and we plan to extend the module to now predict other statuses like AFK or leaving for the day. 

Future Works

Apart from extending the module to predict update status like AFK and leaving for the day.  We can also introduce the concept of reinforcement learning to help models adapt to changes over time and improve their performance.