Show HN: OpenLLMetry – OpenTelemetry-based observability for LLMs https://ift.tt/hvQneb8

Show HN: OpenLLMetry – OpenTelemetry-based observability for LLMs Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://ift.tt/sqAGeMp . There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the gen-AI space so we think it’s the right time to define an open protocol for observability. So we built OpenLLMetry. It extends OpenTelemetry and provides instrumentations for LLM-specific libraries which automatically monitor and trace prompts, token usage, embeddings, etc. Two key benefits with OpenTelemetry are (1) you can trace your entire system execution, not just the LLM (so you can see how requests to DBs, or other calls affect the overall result); and (2) you can connect to any monitoring platform—no need to adopt new tools. Install the SDK and plug it into Datadog, Sentry, or both. Or switch between them easily. We’ve already built instrumentations for LLMs like OpenAI, Anthropic and Cohere, vector DBs like Pinecone and LLM Frameworks like LangChain and Haystack. And we’ve built an SDK that makes it easy to use all of these instrumentations in case you’re not too familiar with OpenTelemetry. Everything is written in Python (with Typescript around the corner) and licensed with Apache-2.0. We’re using this SDK for our own platform (Traceloop), but our hope is that OpenLLMetry can evolve and thrive independently, giving everyone (including our users) the power of choice. We’ll be working with the OpenTelemetry community to get this to become a first-class citizen of OpenTelemetry. Would love to hear your thoughts and opinions! Check it out - Docs: https://ift.tt/8smGpgq Github: https://ift.tt/qrWXV5z https://ift.tt/sqAGeMp October 11, 2023 at 08:10PM

Komentar

Posting Komentar

Postingan populer dari blog ini

Show HN: Interactive exercises for GNU grep, sed and awk https://ift.tt/OxeFwah

Show HN: Interactive exercises for GNU grep, sed and awk Hello! For the past few months, I've been using a Python framework called Textual to create TUI apps for interactive exercises. Released the app for GNU awk earlier today, so thought I'd create a post here. If you already know how to manage Python packages, you can use the following command to get all the three apps: pip install grepexercises sedexercises awkexercises `pipx` should also work, but I haven't tested it. The GitHub repo has the source code as well as more detailed installation instructions. You can use alternative CLI tools to solve these exercises as well. For example, Perl instead of GNU awk or ripgrep instead of GNU grep and so on. Hope you find these TUI apps useful. I'd highly appreciate your feedback. Happy learning :) https://ift.tt/Hp9kFEi August 17, 2023 at 05:13PM
Baca selengkapnya

Show HN: My Book Bulletproof TLS and PKI (Second Edition) Is Out https://ift.tt/5PZ9mxF

Show HN: My Book Bulletproof TLS and PKI (Second Edition) Is Out Hello HN. I am excited to share with you my new book, Bulletproof TLS and PKI. I've worked in this space since the very early days (think SSLv2), always frustrated with the fact that the field is vast but the documentation poor. That first led me to create SSL Labs (which ended up being very popular) and then the first edition of my book (in 2014), where I aimed to cover everything a curious person needed to know about SSL/TLS and PKI. Most importantly, it's a very practical book that you can use to just learn what you need at that moment. The second edition (just out) adds coverage of TLS 1.3. I publish two chapters as a separate (and free) OpenSSL Cookbook. There's another free sample chapter as well. The best part of Bulletproof TLS and PKI is that it's a living book. There's nothing worse than obsolete documentation! Because none of the traditional publishers were interested in that sort of thing,
Baca selengkapnya