Over the last 20 years Qualys has built a highly scalable cloud platform. Today the platform handles over 9 petabytes of data, indexing more than 8 trillion data points on our Elastic Search clusters, moving billions messages a day across our Kafka bus.
Like most cloud-native solutions built in the early 2000’s, early versions of the Qualys Cloud Platform were built as a single monolithic application. This worked well for our initial use cases, but as the platform scaled to handle petabytes of data, we needed a modern microservices-based architecture to meet our customer needs.
As a result, over the last few years, the Qualys platform team has invested heavily in moving away from a single monolithic application to a microservices-based architecture, which has multiple advantages for our customers. First, it results in significant performance enhancements for our end users. And from an engineering perspective, it allows us to do microservice-dedicated releases without impacting the rest of the platform.
One of the key capabilities within the Qualys Cloud Platform is the ability to manage Assets, Asset Groups and User Scoping. Asset Groups are logical groups of host assets, domain assets, and scanner appliances. Organizing assets in logical groups makes it easy to scan a subset of assets, making the Results, Reports and Remediation tasks much more manageable.
But the current solution to manage Asset Groups in Qualys Cloud Platform comes with its own challenges.
Asset Group Management Challenges
As the usage of Asset Groups increased to thousands of asset groups in a single subscription (in some cases more than 20k asset groups), it impacted the customer experience across a wide variety of use cases. For example, pages would load slowly, adding/removing IPs would take longer, and we observed other downstream effects across scanning, reporting and other workflows.
To address these challenges, Qualys is happy to announce Asset Group Management Service (AGMS), the new dedicated backend service to manage asset groups which will start to roll out to our customers starting August 15, 2021.
What is Asset Group Management Service (AGMS)?
AGMS is a new dedicated microservice which offers significant performance improvements for common asset management tasks such as Add, Edit, Delete and Get assets from an asset group.
AGMS offers multiple benefits to our customers. AGMS uses an optimized User Scoping algorithm to find out the affected Asset Groups for the User resulting in efficient Asset Edit operations in Asset Groups and the License Container. AGMS also uses an in-memory data grid which caches all data, reducing the DB calls and network I/O. As a result of this AGMS can handle millions of assets very easily.
Here are some additional highlights:
- Improved performance in Asset Management & Asset Group Management functionality
- Reduces data inconsistencies scenarios from current model and avoids them in the future
- Performance of Asset Tagging functionality improved by 15-20%
Based on insights from our internal benchmarks and feedback from early adopters we have seen performance improvements anywhere from 10x to 30x for common asset group operations, such as adding IPs in bulk, editing Asset Groups and many other common use cases.
These performance improvements are driven by following changes behind the scenes.
- In-memory database
- Distributed processing
- Efficient data structures
- Redesigned database schema
- New APIs optimized for performance.
Please refer to the AGMS online help for details regarding all the UI and API changes made as part of the new AGMS system.
Efficiency at Scale
AGMS is one of many new microservices coming to the Qualys Cloud Platform to significantly enhance performance, scale the platform, and provide additional improvements to our customers. In addition to AGMS, Qualys will roll out other microservices such as Vulnerability Management Scan Processing (VMSP), Vulnerability Management Reporting Service (VMRS) which will use AGMS to fetch asset information and user asset scope.
Frequently Asked Questions
When will the AGMS functionality rollout begin?
We have already deployed AGMS to 30 plus customer subscriptions across all shared platforms. We will start mass rollouts a month after this blog notification. We will start with one Shared Cloud Platform at a time and Qualys Support will notify customers about migration timelines ahead of time.
How can I verify AGMS service is enabled for my subscription?
After AGMS is enabled, the “Host Assets” tab on the Assets page will appear as “Address Management,” and AGMS Help will display in the online help in your subscription.
If you have any questions or comments, please don’t hesitate to reach out to your TAM or Qualys Support.