New top story on Hacker News: Show HN: Xorq – open-source Python-first Pandas-style pipelines

Show HN: Xorq – open-source Python-first Pandas-style pipelines
8 by secretasiandan | 0 comments on Hacker News.
Hi HN, Dan, Hussain and Daniel here… After years of struggling with data pipelines that worked in notebooks but failed in production, we decided to do something about it. We created xorq to eliminate the constant headaches of SQL/pandas impedance mismatch, runtime debugging, wasteful recomputations and unreliable research-to-production deployments that plague traditional pandas-style pipeline workflows. xorq is built on Ibis and DataFusion. We’d love your feedback and contributions. xorq is [Apache 2.0 licensed]( https://ift.tt/KV3JZY6 ) to encourage open collaboration. Repo : https://ift.tt/Zuk49Ep Docs : https://docs.xorq.dev Roadmap Issues : https://ift.tt/Zuk49Ep You can get started `pip install xorq`. Or, if you use nix, you can simply run `nix run github:xorq-labs/xorq` and drop into an IPython shell. Demo video: https://youtu.be/jUk8vrR6bCw Here are some vignettes to look into next: 1. MCP Server + Flight + XGBoost: https://ift.tt/s4F1NHv 2. 1 DuckDB + 2 Writers + 1 Reader: https://ift.tt/cg7XnGs 3. OpenAI UDF: https://ift.tt/bN9IlGF Some features to note: - Ibis-based multi-engine expression system: effortless engine-to-engine streaming - Cache expressions with `.cache` operator - Portable DataFusion-backed UDF engine with first class support for pandas dataframes - Serialize Expressions to and from YAML - Easily build Flight end-points by composing UDFs thanks for checking this out, and we’re here to answer any questions!

Comments

Post a Comment

Popular posts from this blog

North Korea test fires two missiles month before deadline for US to respond on talks

Image
North Korea fired two "unidentified projectiles" on Thursday, Seoul said, as nuclear talks between Pyongyang and Washington remain deadlocked. The projectiles were fired eastwards from South Hamgyong province and came down in the Sea of Japan, also known as the East Sea, South Korea's Joint Chiefs of Staff (JCS) said. They added that the launch, the latest in a series by Pyongyang, was carried out at 16:59 pm local time - or the early hours on the east coast of the United States, during Thanksgiving, one of the country's biggest annual holidays. It was also one day short of the two-year anniversary of the North's first test of its Hwasong-15 intercontinental ballistic missile, which analysts say is capable of reaching the entire US mainland. Pyongyang is banned from firing ballistic missiles under UN Security Council resolutions, and Prime Minister Shinzo Abe said that Thursday's launch was the latest in a series of violations.  "North Korea's repeate...
Read more

New top story on Hacker News: Show HN: Linen – Make your Slack community Google-searchable

Show HN: Linen – Make your Slack community Google-searchable 10 by cheeseblubber | 3 comments on Hacker News. Hi HN! Kam here. I’m the founder of Linen.dev https://linen.dev , a website that makes your public Slack community Google searchable. Linen will sync your Slack threads and make it SEO friendly so your community can find Slack content that was previously hidden. Previously I worked on a popular open source project which had a sizable Slack community. Slack was great for engaging with community members and with early sales. However as community scales Slack becomes this black hole where context becomes lost. Most public communities can’t afford to pay for several hundred/thousand members so they are limited to 10,000 free messages. You run into the problem of people asking repeat questions and not searching in Slack. It also doesn’t help that the Slack UX encourages posting and not searching. We experimented with Github discussions and Discourse but didn’t want another channe...
Read more

Beijing 'preparing tanks at Hong Kong border', warns Trump as protesters clash with police at airport

Image
Protesters clashed with riot police at Hong Kong's international airport on Tuesday evening after flights were disrupted for a second day, as Donald Trump warned that China is moving troops to the border. The airport scuffles broke out in the evening between police and protesters, after protesters allegedly detained two men suspected of being undercover Chinese officials. Trouble began as about 10-15 regular police officers entered the airport without riot gear to assist paramedics after a man collapsed. The man was accused by demonstrators of being a member of Chinese state security. Protesters then drove the police out of the terminal building. Shortly after, about 50 riot police arrived and clashes broke out in and around the entrance of the airport. Police used pepper spray and made a handful of arrests as scenes briefly turned violent. A policeman was cornered and beaten with his own baton before protesters dispersed when he drew his pistol. Cameramen and p...
Read more