Very cool - a lot of well thought out stuff in there.
One area for exploration is letting people turn natural language questions into non-LLM queries, UIs, & dashboards. In other words to let non-engineers codify their questions into queries they can review for correctness and then take the LLM out of the picture.
Imagine if your CEO could ask natural language questions, build their own dashboard, review the generated queries for correctness, and be able to see deterministic results on any metric they care about - without having to ask an intern and without a multi-hour turnaround while it’s implemented.
Codification is kind of the best of both worlds and the underlying idea (explore with an LLM & then codify into something fast and deterministic when ready) is quite universal.
This was too perfect of a setup, had to record a video[0] showing how we do this.
Yes, you definitely need need for a codification layer.
I think a semantic layer is the best way to do that for analytics. Having an LLM write bespoke SQL to answer every question will fail fast.
e.g. if you ask for "revenue by month" against a Snowflake warehouse with hundreds of tables, you are guaranteed to get different answers over multiple attempts.
We[1] use an agent to build a semantic layer over time at Definite so you get consistent results.
This is incredibly cool! A lot of times the user query can be ambiguous enough to make it consistent across runs. The semantic layer is essential to reduce ambiguity, either built by AI or engineers.
That's something we are building! We hope to enhance the report generation as a dashboard builder. Instead of automatically compose an article out of the exploration, we could add more instructions and UI to allow user to specify how different components (vis, data, questions) should be put together to "codify" into a live document to share.
Pretty cool. I like the local install option.
I almost skipped this as more AI wrapper shovelware. Would benefit from putting "Microsoft" in the title.
That's a good suggestion :)
Very cool - a lot of well thought out stuff in there.
One area for exploration is letting people turn natural language questions into non-LLM queries, UIs, & dashboards. In other words to let non-engineers codify their questions into queries they can review for correctness and then take the LLM out of the picture.
Imagine if your CEO could ask natural language questions, build their own dashboard, review the generated queries for correctness, and be able to see deterministic results on any metric they care about - without having to ask an intern and without a multi-hour turnaround while it’s implemented.
Codification is kind of the best of both worlds and the underlying idea (explore with an LLM & then codify into something fast and deterministic when ready) is quite universal.
This was too perfect of a setup, had to record a video[0] showing how we do this.
Yes, you definitely need need for a codification layer.
I think a semantic layer is the best way to do that for analytics. Having an LLM write bespoke SQL to answer every question will fail fast.
e.g. if you ask for "revenue by month" against a Snowflake warehouse with hundreds of tables, you are guaranteed to get different answers over multiple attempts.
We[1] use an agent to build a semantic layer over time at Definite so you get consistent results.
0 - https://www.loom.com/share/2da829dd440e489a8f7e3906c7083048
1 - https://www.definite.app/
This is incredibly cool! A lot of times the user query can be ambiguous enough to make it consistent across runs. The semantic layer is essential to reduce ambiguity, either built by AI or engineers.
That's something we are building! We hope to enhance the report generation as a dashboard builder. Instead of automatically compose an article out of the exploration, we could add more instructions and UI to allow user to specify how different components (vis, data, questions) should be put together to "codify" into a live document to share.
Hyped to use your product in our Mumbai SaaS startup sir!
Feel free to submit requests in github for any customization needs!