Convert API Log Data into Actionable Information

Your API Logs Contain a Wealth of Actionable Information

The amount of valuable information you can obtain from your API logs may amaze you. For example, you can use API log data to answer a multitude of questions about your API, such as:

  • What are the request and response payloads being sent?
  • Which API version is used the most?
  • Where is the API experiencing higher latency?
  • What is the sequence of transactions leading to errors on POST /*/count/events?
  • Which SDKs are used to access the API the most?
  • What are the top endpoints used by each of my customers?
  • Who are my top customers by API usage?
  • Which users are scraping large amounts of data from the API?
  • Which customers are running into 401 unauthorized errors?
  • What is the API activity for a specific customer?

You Probably Pipe Your Logs into Kibana

If you want to gain insights from your API log data, then you’re probably piping those logs into Kibana. If you’re not familiar, Kibana is one of the most popular, if not the most popular, tools for log visualization. It’s part of the Elastic Stack (formerly ELK Stack), which consists of three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine built on Apache Lucene. Logstash is a server-side data processing pipeline. And Kibana lets you explore and visualize Elasticsearch data.

Pros and Cons of Kibana for API Logs and Metrics

If you’ve built an API product, you may already use Kibana to gain insights into its performance or how developers use it. Perhaps you’re considering using Kibana because of its popularity when it comes to log visualization. Either way, it helps to know some of the pros and cons of using Kibana for API logs and analytics.

  • Open source and free to use.
  • Great at visualizing API logs.
  • You can explore massive volumes of log data.
  • Many useful features. Although it should be noted that some features are available separately and some are currently experimental or in beta.
  • Layered on top of Elasticsearch (also a con), making it ideal for use on high-cardinality, high-dimension log data- a must-have feature for API logs.
  • Compatible with Elasticsearch and Logstash only. If you want to use Kibana with other databases, you’re out of luck.
  • Designed for infrastructure metrics and not specifically for API products. You have to customize Kibana for API log use cases.
  • To connect user behavior and API activity into a single journey is incredibly manual and prone to errors over time.
  • You need to use the Kibana Query Language (KQL) or Lucene query syntax (Kibana legacy query language) for queries, so there is a moderate to high learning curve.
  • Maintaining the Elastic Stack takes a lot of effort. For example, you must periodically upgrade each part of the stack and make sure upgrades won’t break any plugins you’re using or require that you rewrite any of your visualizations.

Move Beyond Kibana and Get Actionable Insights from Your API Logs with Moesif

Moesif is more than log visualization, we extract actionable information from your API logs and we guide you as to what to do with it. Moesif is a user behavior API analytics and monitoring service that features a fast query engine and structured event-based data. We designed our platform for API products, so we support a wide variety of API protocols including REST, GraphQL, JSON-RPC, Hypermedia (HATEOAS), and SOAP.

  • High-cardinality, high-dimension API metrics that are compatible with any database, not just Elasticsearch.
  • Automatic analysis of REST and GraphQL APIs.
  • Automatic insights on query parameters and HTTP text payloads like JSON and XML.
  • Track API calls, user actions and behavior, and website activity.
  • Embeddable API logs and charts.

Understand What Your API Log Data is Telling You

Extracting data from your API logs won’t help much if you don’t understand what that data is telling you. Both Kibana and Moesif provide custom dashboards that help you make sense of the data obtained from API logs. However, unlike Kibana, Moesif features pre-built dashboards designed specifically for API products as well as embed templates and public share links. You can use our pre-built dashboards or build your own custom dashboards to gain a better understanding of what your API log data is telling you.

  • Recent API errors
  • HTTP status requests
  • Product usage
  • Daily active users (DAU)
  • Most active users

Don’t Miss Out on Valuable Insights

You need to move beyond log visualizations and towards user behavior API analytics so that you can gain actionable insights from your API log data.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Adam DuVander

Adam DuVander

With APIs and people, anything is possible. Mostly it’s the people.