Menu

Dimensional Data

generative AI

Diffblue Integrates Generative AI-Based Testing Platform With GitLab

Noutați

Diffblue this week generally made available an integration between its automated unit testing platform for Java and the DevSecOps platform from GitLab. The Diffblue Cover platform makes use of generative artificial intelligence (AI) to automatically create unit tests based on reinforcement learning technologies that don’t rely on LLMs—avoiding their drawbacks of sometimes introducing hallucinations and also requiring human review. Diffblue CEO Mathew Lodge said the integration with the continuous integration capabilities found in the premium and ultimate editions of the GitLab platform would, for example, streamline regression testing in a way that should ultimately improve both application quality and security. Diffblue Cover, for example, can update tests as much as 250 times faster than a human developer can write them manually without developer review. That approach also serves to reduce the level of friction many DevSecOps teams encounter when bottlenecks involving testing processes emerge, noted Lodge. The overall goal is to make it simpler for developers to test as much of their own code as possible before integrating it into a build, he added. Otherwise, developers will get fed up because testing is continuously breaking the build, noted Lodge. Ultimately, instead of having to write unit tests, developers become supervisors of a platform that automatically generates them on their behalf, said Lodge. The job of a developer doesn’t go away, but it does fundamentally change, he added. To achieve that goal, developers need to be able to access a platform that writes the tests and can then also execute them automatically. If it takes too long to create the test, chances are high that most developers won’t run it. On average, writing and evaluating a single unit test can take a developer 10 minutes. Over the course of any project, thousands of tests need to be written, so the amount of time testing takes away from coding is often much greater than most IT leaders fully appreciate. Automating those tests should improve developer productivity as more time is available to focus on writing code rather than testing. That doesn’t necessarily eliminate the need for a dedicated testing team, but it does mean that more tests will be run without slowing down the overall pace of application development. Developers naturally want to be able to test code at the very instant they create it. AI platforms can make that happen by, for example, employing reinforcement learning to write unit regression tests. Most developers are not going to resist assuming more responsibility for testing if the tools to automate that task are more accessible. Instead of merely shifting responsibility for testing left toward developers, DevOps teams need to find ways to streamline the process. Otherwise, testing just becomes one in a series of tasks that are being shifted to developers in ways that many of them are coming to resent. It may take some time before AI is fully integrated into software engineering but it’s clear with each passing day more previously manual tasks are being automated. Among the lowest hanging fruit for applying AI to software engineering are clearly testing processes that, if truth be told, few indeed enjoy conducting.

Read More

SmartBear Acquires Reflect to Gain Generative AI-Based Testing Tool

Noutați

SmartBear this week revealed it has acquired Reflect, a provider of a no-code testing platform for web applications that leverages generative artificial intelligence to create and execute tests. Madhup Mishra, senior vice president of product marketing at SmartBear, said the platform Reflect created will initially be incorporated into the company’s existing Test Hub platform before Reflect’s generative AI capabilities are added to other platforms. Reflect provides access to a natural language interface to create tests using multiple large language models (LLMs) that it is designed to invoke. It can also understand the intent of a test to understand what elements to test regardless of whether, for example, a button has been moved from one part of a user interface to another, said Mishra. Test step definitions, once approved, can also be automatically executed using scripts generated by the platform. SmartBear has no plans to build its own LLMs, said Mishra. Rather, the company is focusing its efforts on providing the tools and prompt engineering techniques needed to effectively operationalize them, he added. Reflect is the tenth acquisition SmartBear has made as part of an effort to provide lightweight hubs to address testing, the building of application programming interfaces (APIs) and analysis of application performance and user experience. Last year, the company acquired Stoplight to gain API governance capabilities. Rather than building a single integrated platform, the company is focused on providing access to lightweight hubs that are simpler to invoke, deploy and maintain, said Mishra. The overall goal is to meet IT teams where they are versus requiring them to adopt any entirely new monolithic platform that requires organizations to rip and replace every tool they already have in place, he said. There is little doubt at this point that generative AI will have a profound impact on application testing in a way that should ultimately improve the quality of the applications. As the time required to create tests drops, more tests will be run. Today, it’s all too common for tests not to be conducted as thoroughly as they should be simply because either a developer lacked the expertise to create one or, with a delivery deadline looming, they simply ran out of time. Naturally, the rise of generative AI will also change how testing processes are managed. It’s not clear how far left generative AI will push responsibility for testing applications, but as more tests are created and run, they will need to be integrated into DevOps workflows. Of course, testing is only one element of a DevOps workflow that is about to be transformed by generative AI. DevOps teams should already be identifying manual tasks that can be automated using generative AI as part of an effort to further automate workflows that, despite commitments to automation, still require too much time to execute and manage. Once identified, DevOps teams can then get a head start on redefining roles and responsibilities as generative AI is increasingly operationalized across those workflows.

Read More

Swimm Adds Generative AI Chat Tool for Documentation

Noutați

Swimm today announced it has added a chat tool that enables developers to use natural language to surface insights into code. Company CEO Oren Toledano said the /ask Swimm tool makes it simpler to launch queries that enable developers to better understand how code was constructed. The /ask Swimm tool aggregates documentation along with other related data to surface factors that are not evident in the code itself, such as documentation of business decisions, product design considerations and decisions concerning why specific architectural choices were made. It automatically captures and updates code-related knowledge to provide a continuous feedback loop as code, documentation, files and repositories are created and updated to provide deeper insights into code that goes beyond what might have been actually documented. The overall goal is to provide a mechanism that enables developers to understand how and why code was constructed within the context of an integrated development environment (IDE), said Toledano. Swimm has previously made available a platform that uses generative AI platform to create a static analysis of documentation. That capability should make it simpler for organizations to track documentation at a time when generative AI platforms such as ChatGPT are exponentially increasing the amount of code being written. In theory, those platforms should be able to also create documentation for that code, but there will still be a need for tools to track and analyze it. In the longer term, Swimm plans to continue to extend its usage of generative AI to provide deeper levels of insights across entire applications and software ecosystems to address that challenge, noted Toledano. In the short term, however, it’s clear that AI is making developers more productive than ever. In fact, the volume of code being generated might soon overwhelm existing DevOps pipelines and workflows. Most DevOps teams will need to revamp those workflows as the pace at which more applications than ever are being developed and deployed faster than ever. It’s still early days as far as generative AI adoption is concerned, but it’s clear many developers are already using it to write code. How much of that code makes it into a production environment is difficult to determine, but a lot of that code is going to be of varying quality. A general-purpose AI platform such as ChatGPT was trained using code collected from all across the web. The code generated by these platforms is only as good as the examples used to train it, all of which were created by human developers who may have made mistakes, such as including code that has known vulnerabilities. Regardless of whether code was generated by a machine or a human, the probability that documentation will need to be analyzed as part of a code review process is always high. The challenge and the opportunity is to determine the best way to apply AI to a longstanding challenge that is now being exacerbated by the existence of AI tools that, with each passing day, are only becoming more accessible to developers.

Read More

Best of 2023: Copilots For Everyone: Microsoft Brings Copilots to the Masses

Noutați

As we close out 2023, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023. Microsoft has been doing a lot to extend the coding ‘copilot’ concept into new areas. And at its Build 2023 conference, Microsoft leadership unveiled new capabilities in Azure AI Studio that will empower individual developers to create copilots of their own. This news is exciting, as it will enable engineers to craft copilots that are more knowledgeable about specific domains. Below, we’ll cover some of the major points from the Microsoft Build keynote from Tuesday, May 23, 2023, and explore what the announcement means for developers. We’ll examine the copilot stack and consider why you might want to build copilots of your own. What is Copilot? A copilot is an artificial intelligence tool that assists you with cognitive tasks. To date, the idea of a copilot has been mostly associated with GitHub Copilot, which debuted in late 2021 to bring real-time auto-suggestions right into your code editor. “GitHub Copilot was the first solution that we built using the new transformational large language models developed by OpenAI, and Copilot provides an AI pair programmer that works with all popular programming languages and dramatically accelerates your productivity,” said Scott Guthrie, executive vice president at Microsoft. However, Microsoft recently launched Copilot X, powered by GPT-4 models. A newer feature also offers chat functionality with GitHub Copilot Chat to accept prompts in natural language. But the Copilot craze hasn’t stopped there—Microsoft is actively integrating Copilot into other areas, like Windows and even Microsoft 365. This means end users can write natural language prompts to spin up documents across the Microsoft suite of Word, Teams, PowerPoint and other applications. Microsoft has also built Dynamics 365 Copilot, Power Platform Copilot, Security Copilot, Nuance and Bing. With this momentum, it’s easy to imagine copilots for many other development environments. Having built out these copilots, Microsoft began to see commonalities between them. This led to the creation of a common framework for copilot construction built on Azure AI. At Build, Microsoft unveiled how developers can use this framework to build out their own copilots. Building Your Own Copilot Foundational AI models are powerful, but they can’t do everything. One limitation is that they often lack access to real-time context and private data. One way to get around this is by extending models through plugins with REST API endpoints to grab context for the tasks at hand. With Azure, this could be accomplished by building a ChatGPT plugin inside VS Code and GitHub Codespaces to help connect apps and data to AI. But you can also take this further by creating copilots of your own and even leveraging bespoke LLMs. Understanding The Azure Copilot Stack Part of the Azure OpenAI service is the new Azure AI Studio. This service enables developers to combine AI models like ChatGPT and GPT-4 with their own data. This could be used to build copilot experiences that are more intelligent and contextually aware. Users can tap into an open source LLM, Azure OpenAI or bring their own AI model. The next step is creating a “meta-prompt” that provides a role for how the copilot should function. So, what’s the process like? Well, first, you […]

Read More

PagerDuty Previews Generative AI Copilot for ITSM

Noutați

Under an early access program, PagerDuty, Inc. is making available a tool that brings generative artificial intelligence (AI) capabilities to its IT service management (ITSM) platform. Jonathan Rende, senior vice president of products for PagerDuty, said PagerDuty Copilot for the PagerDuty Operations Cloud extends previous investments in machine learning algorithms the company has made as part of an ongoing effort to apply AI to ITSM. Designed to be invoked via Slack, PagerDuty Copilot makes use of multiple large language models (LLMs) to automate tasks ranging from providing summarization of IT incidents to creating code to automate workflows. PagerDuty plans to transparently swap LLMs in and out of its platforms as AI advances continue to be rapidly made, noted Rende. PagerDuty Copilot provides a level of abstraction for invoking AI models along with appropriate guardrails that make it simpler to manage IT operations without IT teams needing to have AI expertise, said Rende. The overall goal is to boost the productivity of IT operations teams by eliminating much of the drudgery and toil that conspires to make working in IT tedious, noted Rende. AI technologies are not likely to replace the need for IT personnel as much as they will enable IT teams to focus on tasks that add more value to the business, he added. It’s now only a matter of time before generative AI capabilities are pervasively applied across both ITSM and DevOps workflows. Less clear is the impact those capabilities will have on the best practices currently relied on to manage those workflows as more tasks are automated. Ultimately, however, AI should make it easier for more organizations to embrace those best practices as the level of skill and expertise required to manage IT at scale is reduced. In addition, the whole concept of issuing tickets to manage tasks tracked by a central system of record may need to evolve simply because AI has automated requests for service. There will naturally need to be some system of record for tracking requests. Still, ultimately that process will be managed via copilots rather than by a ticket created by an end user that is then tracked via an ITSM platform. Savvy IT teams, in the meantime, are already moving to determine which tasks and workflows will be automated in anticipation of AI becoming more widely embedded in ITSM and DevOps platforms. Roles and responsibilities within IT teams will inevitably evolve as AI increasingly automates mainly mundane tasks, such as creating reports that many IT professionals would rather not spend time writing. The biggest IT management platform challenge in the future might not necessarily be adjusting to AI as much as it will be orchestrating requests that are likely to be generated by multiple types of copilots that have been embedded into almost every application. One way or another, AI is about to transform how IT operations are managed. As disruptive as those advances will be, AI, more importantly, will also enable organizations to manage IT at levels of scale that were previously unimaginable.

Read More

Coexisting With AI: The Future of Software Testing

Noutați

If 2023 was the year of artificial intelligence (AI), then 2024 is going to be the year of human coexistence with the technology. Since the release of Open AI’s ChatGPT in November 2022, there has been a steady stream of competing large language models (LLMs) and integrated applications for specific tasks, including content, image processing and code production. It’s no longer a question of if AI will be adopted; we have moved on to the question of how best to bring this technology into our daily lives. These are my predictions for the software quality assurance testing industry for 2024. Automated testing will become a necessity, not a choice. Developers will lean heavily on AI-powered copilot tools, producing more code faster. That means huge increases in the risk profile of every software release. In 2024, testers will respond by embracing AI-powered testing tools to keep up with developers using AI-powered tools and not become the bottleneck in the software development life cycle (SDLC). The role of the tester will increase and evolve. While AI is helping software engineers and test automation engineers produce more code faster, it still requires the highly skilled eye of an experienced engineer to determine how good and usable the code or test is. In 2024, there will be a high demand for skilled workers with specific domain knowledge who can parse through the AI-generated output and determine if it’s coherent and useful within the specific business application. Although this is necessary for developers and testers to start trusting what the AI generates, they should be wary of spending inefficient amounts of time constructing AI prompts, as this can ultimately lead to decreased levels of performance. For instance, a developer could easily spend most of their time validating the AI-generated output instead of testing the release that will be deployed to users. Being able to distinguish between when to rely on AI and when to forego AI’s help will be key to streamlining the workflow. Eventually, we’re going to start seeing AI-powered testing tools for non-coders that focus on achieving repeatability, dependability and scalability so that testers can truly use AI as their primary testing tool and ultimately boost their productivity. The rise of protected, offline LLMs and the manual tester. As enterprise companies show signs they don’t trust public LLMs (e.g., ChatGPT, Bard, etc.) with their data and intellectual property (IP), they’re starting to build and deploy their own private LLMs behind secured firewalls. Fine-tuning those LLMs with domain-specific data (e.g., banking, health care, etc.) will require a great volume of testing. This promises a resurgence in the role of the manual tester as they will have an increasingly important role to play in that process since they possess deep domain knowledge that is increasingly scarce across enterprises. As we stand on the brink of 2024, it is evident that the synergy between artificial intelligence and human expertise will be the cornerstone of software quality engineering. Human testers must learn to harness the power of AI while contributing the irreplaceable nuance of human judgment. The year ahead promises to be one where human ingenuity collaborates with AI’s efficiency to ensure that the software we rely on is not only functional but also reliable and secure. There will likely be a concerted effort to refine these […]

Read More

CircleCI Extends CI/CD Reach to AI Models

Noutați

CircleCI this week revealed it is extending the reach of its continuous integration/continuous delivery (CI/CD) platform to make it simpler to incorporate artificial intelligence (AI) models into DevOps workflows. In addition to providing access to the latest generation of graphical processor units (GPUs) from NVIDIA via the Amazon Web Services (AWS) cloud, Circle CI has added inbound webhooks to access AI model curation services from providers such as Hugging Face and integrations with LangSmith, a debugging tool for generative AI applications and the Amazon SageMaker service for building AI applications. CircleCI CEO Jim Rose said while there is clearly a lot of enthusiasm for incorporating AI models into applications, the processes being used are still immature, especially in terms of automating workflows that include testing of probabilistic AI models. Most AI models are built by small teams of data scientists that create a software artifact that needs to be integrated within a DevOps workflow just like any other artifact, noted Rose. The challenge is that most data science teams have not yet defined a set of workflows for automating the delivery of those artifacts as part of a larger DevOps workflow, he added. DevOps teams will also need to make adjustments to a version control-centric approach to managing applications to trigger pipelines to pull AI software artifacts that exist outside of traditional software repositories. For example, the inbound webhooks provided by CircleCI now make it possible to automatically create a pipeline whenever an AI model residing on Hugging Face changes. It’s still early days as far as the deployment of AI models in production environments is concerned, but there is no doubt generative AI will have a major impact on how software is developed. AI models are a different class of software artifacts that are retrained instead of being updated, a process that occurs intermittently. As such, DevOps teams need to keep track of each time an AI model is being retrained to ensure applications are updated. At the same time, generative AI will also increase the pace at which other software artifacts are being created and deployed. Many of the manual tasks that today slow down the rate at which applications are built and deployed will be eliminated. That doesn’t mean there will be no need for software engineers, but it does mean the role they play in developing and deploying software is about to rapidly evolve. DevOps teams need to evaluate both how generative AI will impact the tasks they manage as well as the way the overall software development life cycle (SDLC) process needs to evolve. Each organization, as always, will need to decide for itself how best to achieve those goals depending on the use cases for AI,  but the changes that generative AI will bring about are now all but inevitable. The longer it takes to adjust, the harder it will become to overcome the cultural and technical challenges that will be encountered along the way.

Read More

New Relic Adds Ability to Monitor AI Models to APM Platform

Noutați

New Relic today added an ability to monitor artificial intelligence (AI) models to its application performance management (APM) platform. Peter Pezaris, senior vice president for strategy and experience for New Relic, said as next-generation applications are built and deployed, it’s apparent most of them will incorporate multiple AI models. New Relic is extending its APM platform to make it simpler to monitor the behavior of those AI models within the context of an application, he added. To achieve that goal, New Relic has added more than 50 integrations with frameworks and AI models to troubleshoot, compare and optimize different prompts and responses to address performance, cost, security and quality issues such as hallucinations, bias, toxicity and fairness. For example, response tracing for large language models (LLMs) can be applied using New Relic agent software to collect telemetry data that can be used to compare how different AI models are performing and responding to queries. Those results will provide immediate full visibility into the models, applications and infrastructure being used to provide complete visibility across the entire AI stack, said Pezaris. That capability is going to prove crucial as developer use a mix of proprietary, open source and custom large language models (LLMs) alongside a range of other types of AI models to build and deploy applications, he added. Organizations are likely to find themselves managing hundreds of AI models that either they or a third party developed. The challenge, as always, is bringing order to a potentially chaotic process that, in addition to wasting resources, represents a significant risk to the business given the potential for regulatory fines to be levied, noted Pezaris. Each organization will need to determine for itself how best to construct workflows spanning data scientists, application developers, software engineers, cybersecurity teams and compliance specialists. Before too long, organizations will find themselves managing hundreds of AI models that might be integrated into thousands of applications. New Relic is essentially making a case for extending an existing APM platform to address that challenge rather than requiring organizations to acquire, deploy and maintain additional platforms. Eventually, in addition to updating AI models, IT teams will find they are being regularly replaced as advances continue to be made at a fast and furious rate. Data science teams are now making AI models based on significantly larger parameters that make previous generations of models obsolete before they can even be deployed in production environments. As a result, operationalizing AI is going to present DevOps teams with major challenges as they look to both tune application performance and ensure the results being generated are accurate and consistent. That latter issue is especially critical in enterprise application environments where the results generated by an AI model can’t vary from one query to the next.It’s still early days in terms of how AI will be applied to applications, but as AI models join the pantheon of artifacts DevOps teams need to manage, application development and deployment are about to become much more complex to manage.

Read More

Microsoft Previews Additional Copilot Tools for Azure

Noutați

At its Ignite 2023 conference, Microsoft this week previewed Copilot tools to simplify the management of the Azure cloud service along with a tool that streamlines the building and deploying of artificial intelligence (AI) applications on the Azure platform. In addition, Microsoft launched Microsoft Copilot Studio, a low-code tool that automates the process of creating data integration plugins and adding custom copilots within the Microsoft Copilot for Microsoft 365 tool that Microsoft previously launched. Microsoft Copilot for Azure leverages large language models (LLMs) to enable IT teams to use natural language to create, configure, discover and troubleshoot Azure services. It also enables IT teams to create complex commands, ask questions and optimize costs. Erin Chapple, corporate vice president for Azure Core at Microsoft, told Ignite attendees that Microsoft, along with a handful of customers, is already using Microsoft Copilot Azure to manage Azure infrastructure. In the long term, it’s clear that Microsoft is moving toward streamlining the building and deployment of AI applications using Azure AI Studio, a framework for invoking the AI models that Microsoft makes available on the Azure platform. The goal is to make it possible for organizations to create their own copilots based on AI models they have trained. It’s still early days in terms of organizations leveraging AI models to build applications, but it’s already apparent that DevOps and machine learning operations (MLOps), along with data engineering and cybersecurity best practices, will need to converge. Microsoft is making a case for Azure AI Studio as the framework that will enable IT organizations to achieve that goal. Of course, Microsoft is not the only provider of IT infrastructure resources with similar ambitions, but thanks to its investments in OpenAI and the acquisition of GitHub, it is furthest along in terms of defining a framework for building AI applications at scale. Last week, GitHub previewed an extension of the Copilot tools it already provides to help developers write code that leverages generative AI to automatically propose an editable plan for building an application based on natural language descriptions typed into the GitHub Issues project management software. Copilot Workspace will generate editable documents via a single click that can be used to create code that developers can then visually inspect. Any errors discovered by application developers or the Copilot Workspace platform can also be automatically fixed. At the same time, GitHub has extended the scope and reach of Copilot Chat to make it simpler for developers to use natural language to discover issues in their code base. Generative AI is already having a massive impact on the rate at which applications are developed, but that code still needs to be reviewed. Chat GPT is based on a general-purpose large language model (LLM) that is trained by pulling in code of varying quality from all across the web. As a result, code generated by the platform might contain vulnerabilities or be inefficient. In many cases, professional developers still prefer to write their own code. Of course, not every programming task requires the same level of coding expertise. In many instances, ChatGPT will generate, for example, a script that can be reused with confidence across a DevOps workflow. There is no shortage of mediocre developers who are now writing better code thanks to tools such as GitHub Copilot, and soon, […]

Read More

Grafana Labs Acquires Asserts.ai to Bring AI to Observability

Noutați

At its ObservabilityCON event, Grafana Labs today announced it has acquired Asserts.ai to automate configurations and customization of dashboards. In addition, the company is previewing an ability to apply artificial intelligence (AI) to incident management to make it simpler to surface the root cause of an issue. Sift is a diagnostic assistant in Grafana Cloud that automatically analyzes metrics, logs and tracing data, while Grafana Incident is a generative AI tool that summarizes incident timelines with a single click, creates metadata for dashboards and simplifies the writing of PromQL queries. Grafana Labs is also making generally available an Application Observability module for Grafana Cloud to provide a more holistic view of IT environments. Finally, Grafana Beyla, an open source auto-instrumentation project that makes use of extended Berkeley Packet Filtering (eBPF), is now also generally available. That tool enables DevOps teams to collect telemetry data for an IT environment from a sandbox environment running in the microkernel of an operating system. That approach makes it simpler to automatically instrument an IT environment, but there are instances where DevOps teams will be managing complex applications that will still require them to collect telemetry data via the user space of an application. Richi Hartmann, director of community for Grafana Labs, said collectively, these additional capabilities will make it simpler to apply observability across increasingly complex IT environments. For example, the AI technologies developed by Assert.ai will make it possible for DevOps teams to start sending data to Grafana Labs that will enable the cloud service to identify the applications and infrastructure being used. AI models will then be able to automatically generate a custom dashboard for that environment that DevOps teams can extend as they see fit, said Hartmann. In general, machine learning algorithms and generative AI are starting to be more widely applied to observability. The ultimate goal is to automatically identify issues in ways that reduce the cognitive load required to manage complex IT environments while also making it easier to launch queries that identify bottlenecks that could adversely impact application performance and availability. It’s not clear to what degree observability tools might eliminate the need for monitoring tools that track pre-defined metrics, but most DevOps teams will likely be using a mix of both for the foreseeable future. In the meantime, IT environments are only becoming more complex as various types of cloud-native applications are deployed alongside existing monolithic applications that are continuously being updated. The challenge is the overall size of DevOps teams is not expanding, so there is a greater need for tools to streamline the management of DevOps workflows. AI will naturally play a larger role in enabling organizations to achieve that goal, but it’s not likely to replace the need for DevOps engineers, said Hartmann. Conversely, many DevOps teams will also naturally gravitate toward organizations that make the tools they need to succeed available. Today, far too many manual tasks are increasing turnover as DevOps teams burn out. Organizations that want to hire and retain the best DevOps engineers will need to invest in AI. Of course, DevOps, at its core, has always been about ruthlessly automating as many manual tasks as possible. AI is only the latest in a series of advances that, over time, continue to make DevOps more accessible to IT professionals of […]

Read More

Abonează-te la noutăți

Primiți noutăți și oferte despre SDLC tools

Close Menu