⚙️ Prevent performance related problems
Metis allows you to view the SQL commands running in your environments, gain insights into performance and potential errors, and determine the safety of deploying changes to production.
Workflow
To prevent the performance problems, Metis needs to understand how your application interacts with the database and how the database executes the query. Conceptually, you need to do the following:
Step 1: Instrument your application to send the REST and SQL traces to an Open Telemetry Collector.
Step 2: Deploy Metis Telemetry Collector. It generates the Execution Plans of the SQL commands and sends them to the Metis platform.
Step 3: Open Metis web app to view the monitored activity and insights.
- View the root of the distributed traces, usually REST commands, generated by your application
- View the SQL commands created in each trace
- View the execution plan and insights
Let’s examine each step in details.
Step 1 - Instrumenting Your Application
Metis needs to understand what happened in your application and in your database. To do that, your application needs to inform Metis about the interaction details and what queries were sent to the database.
Metis uses OpenTelemetry to receive details about your application’s activity. OpenTelemetry is an open industry standard for capturing signals like traces, metrics, and logs. It’s very similar to your logging library and your application (or libraries like web server or SQL driver) most likely supports OpenTelemetry even now. Metis doesn’t own or develop OpenTelemetry. We just use it as a standard medium of communication. You don't need to use OpenTelemetry SDKs and you can just generate the correct payloads in different way. You can learn more about OpenTelemetry in their documentation.
Your application needs to generate REST and SQL traces, and send them to Metis Telemetry Collector using JSON or gRPC. REST traces are used to capture the application activity within a workflow (like “checkout” API). SQL traces are used to extract the SQL statements that were sent to the database so Metis can analyze their performance. You need to enable traces in your web server (like Express.js or ASP.NET) and in your SQL driver (like pg or JDBC). You can send other traces (like filesystem, infrastructure, or hardware) to Metis Telemetry Collector but they will be ignored.
Below, we cover how you can instrument your application.
Option 1: Instrumenting Your Application With Out-Of-Process Auto-Instrumentation
OpenTelemetry provides a mechanism for auto-instrumenting your application with no code changes. You don’t need to change the source code at all. All you need to do is to put OpenTelemetry libraries in your application folder, configure environment variables, and start your application.
This option is supported in various languages like:
Let’s take JavaScript as an example. You need to install dependencies to OpenTelemetry packages, and then set environment variables like OTEL_TRACES_EXPORTER. You don’t need to modify your application code at all. You can see the example in our repository.
This is the recommended solution. If your application (namely your web server library and your SQL driver library) supports out-of-process auto-instrumentation, go this way.
Option 2: Instrumenting Your Application With Auto-Instrumentation From Inside Of Your Application
Auto-instrumentation mechanism from OpenTelemetry can enable instrumentation of various libraries like web servers, ORMs, and SQL drivers. In some languages, you need to trigger the instrumentation configuration manually. You don’t need to configure instrumentation per each library or modify the libraries, but you need to enable the instrumentation manually.
Technologies that use this approach:
Conceptually, you just need to enable OpenTelemetry in your application code right after your application starts, trigger the auto-instrumentation and then all the libraries supporting auto-instrumentation will be enabled. If your web server library and SQL library support auto-instrumentation, then you’re done. You can see the example in our repository or in another repository.
Option 3: Instrumenting Your Application Manually From Inside Of Your Application
If you can’t use auto-instrumentation, then you can enable the OpenTelemetry support for your application with additional libraries. You need to instrument your web server library (to generate the REST trace) and your SQL driver library (to generate the SQL trace). Depending on your stack, you may use built in support from the library or you may need to look for a 3rd party solution.
Examples:
- Gorm or http.Handler in GO
Conceptually, you need to modify your application code to enable OpenTelemetry right after your application starts. You need to manually configure and trigger instrumentation libraries to enable instrumentation of REST and SQL. You can see the example in our repository.
Option 4: Generating Traces Manually From Inside Of Your Application
If your technology stack doesn’t support OpenTelemetry instrumentation, you can build it up yourself. It doesn’t take much to configure it. Sometimes it’s even explain in the documentation of the libraries you use. You most likely need to use “hooks”, “callbacks”, or “logging” mechanisms of your libraries. For instance, Prisma supports hooks that you can use to build traces and spans.
Conceptually, you need to do the following:
- For REST
- You need to build the REST span that contains
http.urlorhttp.routeorhttp.target - You may include
http.status_codeandhttp.method - The trace must have the trace id, span id, span kind with value
2, start time, and end time
- You need to build the REST span that contains
- For SQL
- You need to build the SQL span that contains
db.statement - You may include
db.name - You may include
net.peer.nameornet.peer.ipornet.host.name - The trace must have the trace id, span id, span kind with value
3, start time, and end time
- You need to build the SQL span that contains
If you need to instrument your application this way, do the following:
- Configure OpenTelemetry in your application code right after your application starts (or use out-of-process instrumentation)
- Manually build traces using the “hooks” mechanisms your libraries provide
Keep in mind that you may want to mix approaches. If your web server library supports auto-instrumentation but your SQL driver doesn’t support OpenTelemetry at all, you can use auto-instrumentation to generate REST traces and only manually instrument your SQL driver.
Examples:
Option 5: Sending Traces Manually Outside Of Your Application
You don’t need to send traces from inside of your application. You may do it offline as there is nothing specific in OpenTelemetry we require. As long as you send us the correct payload, we will analyze it correctly.
For instance, you can scrape your logs, extract web calls and SQL statements, and then send them to the collector with CURL like this:
python3 -c "$(cat <<'EOF'
import time;
import inspect;
NOW_EPOCH=(int(time.time() * 1000000000));
ONE_MILLISECOND=1000000;
payload=inspect.cleandoc('''
{
"resourceSpans" : [{
"resource" : {
"attributes" : [{
"key" : "service.name",
"value" : {
"stringValue" : "console"
}
}
]
},
"scopeSpans" : [{
"spans" : [{
"traceId" : "''' + str(NOW_EPOCH) + '''",
"spanId" : "''' + str(NOW_EPOCH + 1) + '''",
"kind" : 2,
"startTimeUnixNano" : "''' + str(NOW_EPOCH) + '''",
"endTimeUnixNano" : "''' + str(NOW_EPOCH + 1000*ONE_MILLISECOND) + '''",
"attributes" : [{
"key" : "http.route",
"value" : {
"stringValue" : "/YOUR_HTTP_ROUTE"
}
}, {
"key" : "http.status_code",
"value" : {
"intValue" : 200
}
}, {
"key" : "http.method",
"value" : {
"stringValue" : "GET"
}
}
]
}, {
"traceId" : "''' + str(NOW_EPOCH) + '''",
"spanId" : "''' + str(NOW_EPOCH + 2) + '''",
"kind" : 3,
"startTimeUnixNano" : "''' + str(NOW_EPOCH) + '''",
"endTimeUnixNano" : "''' + str(NOW_EPOCH + 200*ONE_MILLISECOND) + '''",
"attributes" : [{
"key" : "db.statement",
"value" : {
"stringValue" : "SELECT COUNT(*) FROM table"
}
}
]
}
]
}]
}
]
}'''
);
print(payload);
EOF
)" | curl -H "Content-Type: application/json" -X POST http://localhost:4318/v1/traces -d @-
Conceptually, you can send such a request with whatever technology you want as long as the payload is correct.
Step 2: Deploying Metis Telemetry Collector
Metis Telemetry Collector is distributed as a Docker container. It does the following:
- Receives OpenTelemetry traces
- Connects to the database to get the execution plans
- Sends the execution plans to Metis platform
You need to deploy the collector somewhere where you’re able to do the following:
- Your application must be able to connect to Metis Telemetry Collector to send the OpenTelemetry traces
- Metis Telemetry Collector must be able to connect to your database to send the EXPLAIN query
- Metis Telemetry Collector must be able to connect to Metis public API over the Internet
You can host Metis Telemetry Collector anywhere (locally, in your infrastructure, in your cloud, etc.) as long as you have the required network connectivity to achieve the above. OpenTelemetry recommends running the container near the instrumented application. By default the applications send traces to [http://localhost:4318](http://localhost:4318) and assume that the collector runs on the same host.
Read the documentation to learn how to start using Metis Telemetry Collector.
Step 3: Using Metis
At this point you can use Metis to analyze the performance of your application and how it interacts with the database.
You may also consider integrating your CI/CD with Metis or your production databases for better monitoring.
Frequently Asked Questions
Do you support X?
We support anything that can deliver proper REST and SQL spans. Sometimes you need to add more code around the libraries you use to extract the details (like SQL query parameters) and send them to Metis Telemetry Collector.
Does it work in testing scenarios? Does it work with unit tests?
Yes. However, some testing frameworks do not propagate REST traces properly when you test your API endpoints. They may run your tests outside of OTel context or not capture the details properly. You may need to create spans manually in such a case. This depends on the testing framework you use and how well it integrates with testing your web server library.
Do I need to have the REST span?
At this point, yes. We use the REST span to group SQL statements and present them in the UI. If you are in a scenario where you can't have the REST span, let us know!
My library doesn't integrate with OpenTelemetry. What should I do?
You need to build the OpenTelemetry spans manually. To do that, you need to learn how to extract details from your library. Typically, there are mechanisms like hooks, events, callbacks, middleware, or annotations.
Conceptually, you want to register for an even from your library that explains what happened. In the event handler, you would extract the details (like REST path or SQL query with parameters), and send them to Metis.
Do I have to use OpenTelemetry?
No, you do not. Metis Telemetry Collector uses OpenTelemetry standard for defining the payload shape. However, you can send the payload using any technology that can send JSON or gRPC.
I don't use OpenTelemetry inside my application. What should I do?
The first solution is to install the OpenTelemetry libraries and enable the instrumentation.
The second solution is to capture events from your application manually (via hooks, callbacks, middleware, events, or other mechanism that your libraries support), and send them to Metis Telemetry Collector with JSON or gRPC.
I don't know how to instrument my library. Can you help me with that?
Obviously. Contact us and we will be able to help. We constructed many examples in languages like JavaScript, TypeScript, GO, Java, Kotlin, C#, and Python.
How do I enable OpenTelemetry in low-code/no-code solutions?
Depending on your solution, you may need to extract activities' details manually and send them to Metis.
I have logs from my application. Can I use them with Metis?
Yes, as long as they contain all the details we need. You can process your logs offline and send them to Metis even with CURL.