Trovares xGT: Proof of Concept Considerations for Graph Search Engine
Introduction
In the rapidly evolving field of data analytics, graph-based methods have become increasingly important for uncovering complex relationships and patterns within large datasets. However, traditional graph database solutions often struggle to deliver the necessary performance, especially when dealing with massive datasets. Trovares xGT addresses this challenge with a high-performance, user-friendly graph search engine designed for organizations seeking to gain deep insights without requiring specialized data science expertise.
This white paper outlines the key features and benefits of Trovares xGT, as well as considerations for conducting a Proof of Concept (PoC) to evaluate its potential in your organization.
Trovares xGT: Overview and Value Proposition
Trovares xGT is a software solution that leverages Generational AI to simplify and accelerate the analysis of in-memory property graphs. Unlike many graph databases that require complex query languages and extensive knowledge of graph theory, Trovares xGT is designed to be accessible to non-experts. With its patented technologies, xGT delivers exceptional performance, enabling users to find patterns in large datasets quickly and efficiently.
Key Benefits of Trovares xGT:
Performance: xGT has been benchmarked to answer queries hundreds of times faster than traditional graph databases. This speed advantage allows users to explore data interactively, following trains of thought in real-time without waiting for long query execution times[1].
Ease of Use: Trovares xGT does not require users to learn a new query language or modify existing databases. It integrates seamlessly with existing systems, making it easy to adopt without disrupting workflows.
Scalability: xGT scales from small deployments on laptops to massive server installations capable of handling planetary-scale graphs. The same executable runs on all platforms, ensuring consistency and reliability.
Integration with Existing Workflows: Trovares xGT ingests data from widely-used database systems, such as Oracle, MySQL, MongoDB, Neo4j, and Snowflake, among others. This integration allows organizations to continue using their existing tools while enhancing their graph analytics capabilities with xGT.
Trovares xGT Release 2.0: Enhancing User Experience and Performance
With the release of Trovares xGT 2.0, the product has evolved to include a Desktop browser application that simplifies data ingestion, analysis, and visualization. This no-code environment empowers analysts to perform complex graph analytics without needing programming skills, making advanced graph search accessible to a broader audience.
New Features in Release 2.0:
Trovares Desktop Browser Application: The Desktop application provides a user-friendly interface that allows analysts to ingest data, analyze graphs, and visualize results without writing code. This feature is particularly valuable for organizations that want to enable business users to explore graph data independently.
No-Code Data Analysis: Users can perform analysis on datasets with billions of edges, using an intuitive interface that hides the complexity of graph search from the user. This allows non-technical users to unlock the power of graph analytics.
Quick Deployment: Trovares xGT is distributed in Docker containers, making it easy to deploy and run on a wide range of hardware. Whether on a laptop or a large server, xGT can be up and running in minutes.
High Performance: xGT is optimized to use all available compute cores, driving them to near 100% utilization to deliver fast results. This is particularly beneficial for organizations with high-performance computing environments.
Non-Intrusive Deployment:
Trovares xGT is designed to be a co-processor, complementing existing workflows rather than replacing them. This approach allows organizations to enhance their analytics capabilities without the need for significant changes to their infrastructure. By operating alongside existing tools, xGT boosts productivity while preserving the investments organizations have made in their current systems.
Architecture and Technical Considerations
Trovares xGT is built on a foundation of patented inventions that transform graph pattern searching into an embarrassingly parallel computer science problem. This architectural approach allows organizations to achieve high performance and scalability, even when analyzing vast datasets.
In-Memory Dynamic Graphs
At the core of Trovares xGT is its support for in-memory dynamic graphs, which use strongly-typed graph data structures. This design is crucial for achieving the high performance and scalability that xGT is known for. While xGT is not an ACID-compliant database, it supports all ACID properties except Durability, which contributes to its scalability.
Multi-User Transaction Support
Trovares xGT is designed for multi-user environments, with support for stringent user data access controls. The system allows for both frame-level and row-level data access controls, ensuring that users can only access the data they are authorized to see. This is particularly important for organizations with strict security requirements, such as government agencies and large enterprises.
Rolling Graphs
Trovares xGT supports "rolling graphs," which are continuously updated with new information while older data is aged away. This feature is valuable for organizations that need to maintain up-to-date insights from constantly changing datasets, such as network security logs or financial transaction data.
Scalability
Trovares xGT is highly scalable, capable of running on a wide range of platforms, from laptops to the largest servers with 64TB of shared memory. This flexibility allows organizations to start small and scale up as their needs grow, without having to switch to a different platform.
Integration with Existing Workflows
Trovares xGT integrates seamlessly with existing database systems of record, making it easy for organizations to leverage their existing data infrastructure. The product supports data ingestion from various sources, including:
Databricks
Oracle
MySQL
SAP
MongoDB
Neo4j
Snowflake
Open-Source ODBC Connector
To simplify data ingestion from existing systems, Trovares xGT supports an open-source ODBC Connector that can be launched from a Python Client API. This connector allows organizations to ingest data from systems of record such as Oracle and MongoDB, ensuring that xGT can work with the data that is already in place.
Apache Arrow Endpoint
For easier integration with modern data science tools, Trovares xGT operates as an Apache Arrow Endpoint. This enables direct interconnection with other applications that also use Apache Arrow and supports reading and writing Parquet files. This capability is important for organizations using open-source data science tools, as it allows them to integrate xGT into their existing workflows.
Data Access Controls
Security and data access controls are critical considerations for any organization working with sensitive data. Trovares xGT provides robust data access controls at both the frame and row levels, ensuring that users can only access the data they are authorized to see. This is particularly important for organizations with stringent security requirements, such as government agencies and large enterprises.
Accessor for Data Access Control
All data access in Trovares xGT is managed through an Accessor, which ensures that Cypher queries and Whole Graph Algorithms only access data that the user is permitted to view. This centralized control mechanism simplifies the implementation of security policies and ensures compliance with organizational security regimes.
Proof of Concept (PoC) Considerations
Organizations interested in evaluating Trovares xGT can conduct a Proof of Concept (PoC) to assess its capabilities and determine whether it meets their needs. A PoC can be conducted either on-premises or in the cloud, depending on the organization's preferences.
PoC Components
Ease of Use Testing: The Trovares Desktop application allows non-experts to perform graph analytics, making it easy to evaluate the user-friendly nature of the product. Organizations can test the application's no-code environment to determine whether it meets their needs for ease of use.
Graph Ingest Performance Testing: Trovares xGT is optimized for high-performance data ingestion, particularly when using Parquet files. Organizations can test the speed at which xGT can load large datasets into memory and begin analyzing them. Ingest performance can be measured in terms of the number of attributes loaded per second.
Cypher Query Performance Testing: Trovares xGT supports the Cypher query language, making it easy for organizations to compare its performance with existing solutions. By running standard Cypher queries, organizations can assess how xGT's performance compares to their current tools. Trovares plans to support the new ISO standard GQL graph query language, ensuring compatibility with future developments in graph analytics.
Free Developer Edition
For organizations that want to explore Trovares xGT without committing to a full PoC, a free Developer Edition is available for download. This edition is capped at 8 cores, allowing organizations to test xGT's functionality on a smaller scale. However, for a more comprehensive evaluation, Trovares recommends a formal PoC using the full-performance version of xGT.
Conclusion
Trovares xGT offers a powerful solution for organizations seeking to enhance their graph analytics capabilities without disrupting their existing workflows. With its high performance, ease of use, and seamless integration with existing systems, xGT provides a path to state-of-the-art enterprise-wide graph analytics.
Whether your organization is looking to uncover patterns in large datasets, accelerate query performance, or empower non-experts to perform advanced analytics, Trovares xGT delivers the tools you need to succeed. A Proof of Concept is the ideal way to evaluate xGT's potential in your organization and determine how it can help you achieve your business goals.
Appendix 1: Supreme Trovares xGT Graph Query Performance Over Industry Leader Neo4j
Neo4j Overview
Neo4j is a leading graph database platform renowned for its ability to store and analyze complex relationships between data. Its robust graph analytics capabilities have made it a dominant player in the market, holding an estimated 40-50% share. Neo4j is widely utilized across various industries, including finance and telecommunications, due to its effectiveness in handling intricate network data.
The Study
This study compares the performance of Trovares xGT and Neo4j Enterprise on AWS using two well-known graph query tasks:
·
2-hop Motif: This query pattern examines how one entity connects to another through an intermediary. For example, in a scenario where Ava sends a message to Cat via Ben, this forms a 2-hop connection (see Figure 1a). This pattern illustrates how information or influence can traverse a network even without direct connections
Temporal Triangle Motif: This pattern involves three entities connecting at different times. For instance, Ava communicates with Ben, Ben communicates with Cat, and later Cat communicates back to Ava, forming a time-sequenced triangle (see Figure 2a). This motif highlights how relationships or events evolve over time within a network.
The Results
Tables in Figures 1b and 2b present a comparative analysis of Trovares xGT and Neo4j's performance. The data sizes range from 10K graph edges (second column) to 100M graph edges (sixth column). The tables include the performance metrics for Neo4j and xGT in terms of wall-clock seconds, with the fourth row indicating the speedup factor (calculated as Neo4j’s time divided by xGT’s time). Figures 1c and 2c visually represent the speedup factor of xGT over Neo4j.
Key findings include:
xGT significantly outperforms Neo4j with speedup factors ranging from 6x to 54x for datasets with 10K graph edges.
As the graph size increases from 10K to 100M edges, xGT's speedup factor exceeds 500x.
Conclusion
Our analysis demonstrates that Trovares xGT consistently and progressively outperforms Neo4j as data sizes increase. For further information on these results and additional applications, please contact david@trovares.com.
[1] Refer to Appendix 1 for a performance comparison study of Trovares xGT and Neo4j Enterprise on AWS.
Comments