In this blog post, we discuss metaphactory’s app mechanism that supports the composable enterprise approach and look at the following: the components included in an app, the development-staging-production lifecycle of an app, the architecture of an app and an app example.
The composable enterprise powered by metaphactory
Supporting knowledge workers in reaching scalable, knowledge-driven decisions in their daily tasks requires organizations to implement solutions that drive knowledge democratization and make all of the data that is available to them interpretable and consumable by all relevant people, processes and tools in the enterprise.
Previous approaches based on centralized data lake architectures have tried to consolidate the diverse data sources that organizations have access to, but have not been successful in providing the right level of interpretability that makes data human- and machine-understandable.
The composable enterprise approach aims to solve these challenges by transforming heterogeneous data sources into data products that
- are managed by data owners with respective domain expertise and
- can be combined into a data mesh to support diverse use cases and user information needs.
With metaphactory, we support the management of data products in a semantic knowledge graph, where these data products are linked together through a common semantic layer based on W3C open standards. The semantic layer can consist of multiple, interlinked and complementary ontologies (semantic models) that ensure a well-defined, shared understanding of the data across different sources.
A data product is then a combination of one or more ontologies, vocabularies, and instance datasets that have been prepared and bundled to serve a specific use case. However, because disparate data sources are linked using semantic models, data products become reusable beyond the use cases they were initially created for, across data domains. This helps organizations avoid vendor lock-in and gives them the tools needed to make data FAIR.
Layered on top of the data mesh is a fleet of applications (or apps) that utilize the data products to support business users in specific aspects of their work and are published as a service for user consumption - as depicted in the diagram below. These applications might only publish APIs for integration with existing tools, or might also include use-case-oriented UIs. These are model-driven user interfaces that target specific user groups with specific user interaction patterns and utilize the underlying semantic model and data products.
Composable enterprise architecture powered by metaphactory
The apps are maintained in a well-defined lifecycle that includes mechanisms for rolling out changes to the model or the data sources, as well as versioning and publishing.
In this blog post, we discuss metaphactory’s app mechanism that supports the composable enterprise approach and look at the following:
- the components included in an app
- the development-staging-production lifecycle of an app
- the architecture of an app
- an app example
metaphactory’s app-based environment
A one-stop enterprise knowledge graph platform, metaphactory delivers key functionalities to support the entire lifecycle of building and managing knowledge graphs and targets multiple, key stakeholders:
- Semantic graph modeling (ontology engineering) mainly targeting knowledge graph engineers and knowledge stewards;
- Data management (vocabulary or taxonomy management, as well as graph instance data authoring) mainly targeting knowledge stewards and domain experts;
- Model- and data-driven application development (graph exploration, search interfaces and knowledge discovery), usually done by application engineers for direct consumption by end users.
The metaphactory environment is purposely modular, so as to be able to support multiple use cases through a data mesh architecture as described above. Thus, one metaphactory deployment is able to not only provide the functionality required by each of the above stakeholders, but also to target multiple use cases. It is often the case that one integrated knowledge graph is composed of multiple models, multiple datasets and targeted at different end users with different information needs.
An integral part of the metaphactory platform is therefore the possibility to develop an unlimited amount of custom modular extensions. These extensions may provide additional artefacts, such as application pages, templates and configurations that are required for a distinct use case. An easy and familiar way of bundling such extensions is to package such artefacts into individual apps.
The app-friendly metaphactory environment has the added benefit of being able to be tested and updated individually. Thus, different application engineers can extend the app they are working on, and update and manage versions in a collaborative manner without jeopardising the effectiveness of additional apps and use cases.
App components
In summary, apps allow a single metaphactory environment to support multiple use cases in a modular fashion. Behind each use case, one often finds specific models, or ontologies, that can guide app components to generate model-driven views of data as needed for each specific use case. Therefore, the app components themselves are also often largely model-driven.
metaphactory apps package different kinds of custom platform extensions to serve customer or domain-specific needs:
- End-user application and template pages (the main functionality behind the target use case)
- Configuration: RDF namespaces, system settings, security settings, etc.
- Customizations for the look and feel, like CSS files, images, header and footer resources
- Custom backend JAVA components, like REST services and custom federation service implementations
- Custom Web components to be used in templates and application pages
- Semantic resources: queries, fields or diagrams
By packaging the content in distinct apps, the custom configuration can be easily distributed and deployed in different system instances. Whereas some configuration is tied to each of the standard continuous integration and development environments (development, staging, production), the majority is environment independent and does not require adaptation. The separation between system configuration and other configurations as distinct apps makes it easier to maintain, extend and release successive versions of a metaphactory-based solution.
App lifecycle
Much like other software development lifecycles, developing apps in metaphactory also commonly implies having three environments: 1) development, 2) staging and 3) production - as depicted in the diagram below.
metaphactory app lifecycle - Development, Staging & Production Stages
Feature development and updates are done in the development environment. The staging environment is similar to the production but for the purpose of testing fixes or new versions of apps. Production is the environment where users have access to the final and approved apps.
Given these environments, we often see in metaphactory the development of apps with the following specific goals: use case, branding and config apps.
Use case apps contain use case-specific UIs (i.e., templates) that form the core portion of metaphactory instance and are unchanged between the three different environments. Note that you may have multiple use case apps deployed into a single metaphactory instance.
A branding app is recommended in order to separate the functionality in the use case apps from styling and branding matters (e.g., logos, colour scheme, icons, etc.).
Deployment-specific configuration unique to each environment should be moved to a third independent config app or directly injected as (mounted file) secrets via the container deployment/orchestration framework. These will typically include things like permissions, Single-Sign On configurations, repository configuration, etc., which usually differ between Continuous Integration / Continuous Delivery (CI/CD) environments. The example diagram above shows how three different deployments can share some apps but have distinct deployment-specific configurations containerised within one app.
It is also recommended to manage apps under a source control system, like Git. Version control systems also allow for the development of multiple features, which can be easily separated into branches, and also support tagging and version comparisons. By using CI/CD pipelines, apps can be fetched from the source control system, facilitating deployment updates and the creation of new instances, if needed, for high availability (i.e., with load balancing).
App architecture, volumes and storage
metaphactory apps are packaged as a zip file and are recognised by metaphactory through a plugin.properties file that declares the same ID as the zipped folder (amongst others).
Each app is allocated its own dedicated storage. Although apps can not be deployed through a storage, the storage layer provides an abstraction from the low-level file system and can connect to different storage implementations (local file-based, remote AWS S3 object-based, etc.) in parallel. Every storage is identified by a storage ID. A metaphactory storage fulfills other purposes apart from persisting apps. Therefore, not every storage is dedicated to an app. If no explicit storage type is defined for an app, it is assumed to be file-based.
Once loaded in metaphactory, apps are stored in a dedicated, persistent volume and are therefore preserved following the redeployment of the metaphactory container.
metaphactory app volumes
As shown in the above figure, the apps volume sits alongside the default runtime-data volume and the storage volume (which is used for Git integration for asset management). Internally, the metaphacts platform reads the app artefacts from respective app storages and applies them according to the delegation order specified by the plugin dependencies with the runtime storage having the highest priority. That means that if, for example, a template is found in multiple storages, the one with the highest order will be rendered, with the runtime (i.e., a template that’s been modified at runtime through metaphactory’s editor interface) always taking precedence.
Most often, versioning and collaborative editing for apps is required. Git integration for apps can be enabled through the Git storage layer, which provides an abstraction for accessing app or runtime content from a Git repository. As mentioned previously, Git integration allows for apps to be managed and versioned, which facilitates history, tracking, backup as well as the distribution of apps to different environments using CI/CD pipelines (e.g., development and staging).
App examples
Wikidata app
One example of a metaphactory app is our public Wikidata demonstrator - an online demo system based on the open Wikidata dataset, which showcases key features for search and visualization, as well as federation with additional datasets (such as sensor or weather data), machine learning algorithms, and similarity search functionalities.
The Wikidata app includes several application and template pages which can be adjusted and reused in other environments. For example, the end-user search interface leverages the underlying model and allows end users to combine keyword searches with type-based queries. This app page is available for testing on our GitHub repository.
Resource Hub app
Another great example of a reusable metaphactory app is our Resource Hub which is used by the metaphacts marketing team to manage and expose material such as case studies, blog posts, tutorials, or videos to customers and prospects.
The Resource Hub app includes the schema.org ontology and some extensions thereof to model the data, some controlled vocabularies that cover topics such as Industries or Use Cases, as well as several templates for adding and curating data and searching and exploring content. It is exposed to end users via an integration with the metaphacts website and in the next version, it will also support the publication of structured data to search engines for SEO purposes.
Nobel Prize app
Finally, one other example of a reusable metaphactory app is our Nobel Prize demo system app centred around Nobel Prize laureates and the awards that they have received. It includes the Nobel Prize ontology with some extensions, a special vocabulary and the instance data itself as well as some templates for searching and visualizing data.
Try this yourself
To see how metaphactory’s app mechanism works in practice, you can get started with metaphactory today using our free trial.
If you select the tutorial option, you will already get an example of a pre-loaded metaphactory which includes several template and application pages relevant to the tutorial, as well as a pre-loaded data product containing an ontology, two vocabularies, and a small dataset. The exercises in the tutorial will also guide you through creating your own application pages to extend the pre-loaded app.
