Show HN: Crul – Query Any Webpage or API https://ift.tt/jsQBKNE

Show HN: Crul – Query Any Webpage or API Hi HN, we’re Carl and Nic, the creators of crul ( https://www.crul.com ), and we’ve been hard at work for the last year and a half building our dream of turning the web into a dataset. In a nutshell crul is a tool for querying and building web and api data feeds from anywhere to anywhere. With crul you can crawl and transform web pages into csv tables, explore and dynamically query APIs, filter and organize data, and push data sets to third party data lakes and analytics tools. Here’s a demo video, we’ve been told Nic sounds like John Mayer (lol) ( https://ift.tt/RewS9rz ) We’ve personally struggled wrangling data from the web using puppeteer/playwright/selenium, jq or cobbling together python scripts, client libraries, and schedulers to consume APIs. The reality is that shit is hard, doesn’t scale (classic blocking for-loop or async saturation), and comes with thorny maintenance/security issues. The tools we love to hate. Crul’s value prop is simple: Query any Webpage or API for free. At its core, crul is based on the foundational linked nature of Web/API content. It consists of a purpose built map/expand/reduce engine for hierarchical Web/API content (kind of like postman but with a membership to Gold's Gym) with a familiar parser expression grammar that naturally gets the job done (and layered caching to make it quick to fix when it doesn’t on the first try). There’s a boatload of other features like domain policies, scheduler, checkpoints, templates, REST API, Web UI, vault, OAuth for third parties and 20+ stores to send your data to. Our goal is to open source crul as time and resources permit. At the end of the day it’s just the two of us trying to figure things out as we go! We’re just getting started. Crul is one bad mother#^@%*& and the web is finally yours! Download crul for free as a Mac OS desktop application or as a Docker image ( https://www.crul.com ) and let us know if you love it or hate it. ( https://ift.tt/cQgSmdG ) And come say hello to us on our slack channel - we’re a friendly bunch! ( https://ift.tt/FLKcntT ) Nic and Carl ( https://ift.tt/N0b7coT ) https://www.crul.com/ February 28, 2023 at 11:12PM

Komentar

Posting Komentar

Postingan populer dari blog ini

Show HN: Interactive exercises for GNU grep, sed and awk https://ift.tt/OxeFwah

Show HN: Interactive exercises for GNU grep, sed and awk Hello! For the past few months, I've been using a Python framework called Textual to create TUI apps for interactive exercises. Released the app for GNU awk earlier today, so thought I'd create a post here. If you already know how to manage Python packages, you can use the following command to get all the three apps: pip install grepexercises sedexercises awkexercises `pipx` should also work, but I haven't tested it. The GitHub repo has the source code as well as more detailed installation instructions. You can use alternative CLI tools to solve these exercises as well. For example, Perl instead of GNU awk or ripgrep instead of GNU grep and so on. Hope you find these TUI apps useful. I'd highly appreciate your feedback. Happy learning :) https://ift.tt/Hp9kFEi August 17, 2023 at 05:13PM
Baca selengkapnya

Show HN: Create demos & guides just with a simple prompt https://ift.tt/HfWo3mz

Show HN: Create demos & guides just with a simple prompt Teams often create demos & guides on a daily basis. But creating them is time consuming and needs a lot of effort. From content to the branding, customizations, and many more - they are not something you would want to do on a daily basis. Enter Fable. Purpose built for teams in marketing, sales & customer success. Fable is apt for engaging visitors, closing deals faster, and onboarding your users quickly. https://ift.tt/IVCbkiG October 17, 2024 at 08:32PM
Baca selengkapnya