This is a really cool idea. I love that you expose the personal profile as markdown. Reminds me of this article and exposing the system prompt: https://koomen.dev/essays/horseless-carriages/. Well done!
Thats a great point, and a good articl. I think the example they bring up as "good" and what we are leaning into here, is the idea of transparency and agency of being able to see and modify your profile.
The tension we have been finding is that we dont want to require people to "know how to prompt" to get value out of having a profile, hence our ongoing thinking around how to bootstrap good personal profiles from various data sources.
As Koomen notes, a good profile feels like it could be the best weapon against "AI slop" in a case I want something sharp and specific. But getting to that requires knowing how to prompt most of the time.
Funny, I did the swipe thing and then the first result was this post with a [dive] tag. No idea if cheeky or if it actually got that from my choices, but I had a laugh anyway. Neat PoC!
edit: ooh, I see what the swiping did:
## Analysis of user's tech interest:
The user demonstrates a strong interest in advanced technical topics, particularly in the realm of artificial intelligence, machine learning, and low-level systems programming/security (e.g., kernel exploitation). They are drawn to articles that involve practical application, model creation, and deep dives into complex technical architectures. Their interest in "Show HN" articles suggests an appreciation for new, innovative projects, especially those with a technical or AI focus. They show less interest in general hardware announcements (like new microcontrollers), historical tech accounts, or very niche, non-AI/ML/security-related programming topics.
Yeah, that's pretty much spot on. Wonder if there's a way to match that against the topics I actually commented on, but at a glance it's pretty cool!
Nice, great to see where it nails it, and has the right feel! Using your own comments directly, or in the first instance allowing users to paste in sections of their commented on or upvoted history is a great next step. Not sure yet whether the flow should be in place of or in addition to doing the survey.
I've been wanting this for a while! As an alpha prototype its great. Skip/Skim/Deep dive feels like the right breakdown to me. Having a different color but same UI feels right too.
Other than quality of life stuff (multiple pages for example), I'd like to see it continually learn.
A few things got miscategorized and I'd love for it to naturally correct that with additional input from me.
Hey so cool to hear! the skim/skip/dive mapped to how we use things, but thats sample size of 3 haha.
The idea of having some kind of thumbs up/down on what you see after getting recs, that gets added to your preferences, or being able to do another round of preferences (rather than just re-doing them like we have now) is for sure on our next steps if we continue with. Were not quite sure what the feedback loops will be yet (we did look at adding whole webhistory for example but that felt like a bit much and pretty invasive).
For the miscats, on a meta level what we are generally interested in is whether they come from compression of the preferences into your user profile (essentially if more or better data is the path to better context for such a specific usecase, or whether there is more bang for buck optimizing the various prompts. Keen to hear if its obvious from looking at your profile what was the case.
If we get serious with this evals are a must next step. We are only 2 days in at the moment :)
As it is currently written, this is less useful the more niche your interests are. I think for such users, looking at their comment history or upvoted history might be useful instead of or in addition to just sampling recent, popular articles.
In my case, none of the topics I most like to read about and discuss on HN (package management, software freedom, next-gen CLI tools, next-gen shells, philosophy, desktop Linux, functional programming, hacker history, literate programming, Emacs, bitching about common development practices, programming language design, configuration languages) managed to appear in the 30-post sample I used. The profile it wrote for me was pretty good considering that, but definitely not great.
The assessment was also mistaken about my degree of interest in "low level" technical details like binary file formats (in fact it's rather low, although it has gradually increased over time), and my degree of interest in theoretical computer science issues (in fact it's high, but all of the theoretical papers in the sample were about machine learning, which was not an area of academic focus for me).
I do really like the simplicity and customizability of this (exposing the profile as Markdown and making it editable is awesome), and the quality of the results is very good given the tiny input size. But if your primary interests are not super aligned with the mainstream on HN, you won't get a chance to demonstrate that you like them. If users could type a few terms to say what their biggest interests are before running through the samples, this could work even better for people like me.
It would also be interesting if this could work based on article contents and not just headlines. Sometimes I open something and close it immediately, or I open it undecided as to whether I will skim or read closely.
This is great to hear in such detial. One of the first cabs of the rank to improve this would be greater user control over what preferences to include, and/or smarter selection of the pool to select. This kind of focused preference is super important, especially as i use hackernews, and clearly for you and i suspect others as well.
In fact I would pose that I have a couple of disparate interests or "profiles" that i would like to have greater control over/support in generating, that are non overlapping sets of topics and types of content. The ability to have greater agency in creating them and managing them is something we are keen to explore.
The article comments one is a toughie, as LLM use skyrockets when you scrape and consume content from the links. It would be awesome to include it, but would likely need to be paid, just from a cost perspective.
Really appreciate the detail here, this makes it easier to turn your examples into a test/eval/feature case.
>I have a couple of disparate interests or "profiles" that i would like to have greater control over/support in generating, that are non overlapping sets of topics and types of content
This sounds like a great feature! My appetites for different clusters content certainly vary according to my mood! Perhaps "mood" would actually be a cute-but-clear name for such distinct/multiple profiles. :)
> The article comments one is a toughie, as LLM use skyrockets when you scrape and consume content from the links. It would be awesome to include it, but would likely need to be paid, just from a cost perspective.
Hm. That is a good (and in retrospect, obvious) point. If it makes the feed a lot better, I think it could certainly be worth it for some users. If it only makes a small difference, maybe not. It might be interesting for you to experiment and write about, since what kind of difference it will make isn't obvious (at least to me) up front.
yeah we are thinking a lot right now about good language for how to separate the notion of a "profile" as representative of who i am, from the concept of a profile that represents what mode im currently operating in right now. In the context of browsing, mood actually captures it quite well i agree. As we think wider to the different ways we might want a profile to help guide what content we receive or results we get when using other tools, that language isnt quite so clear yet.
We will have to do some combo of much more internal testing, construct evals, or just capture more info about peoples usage coupled with an ability to provide feedback in order to even get a handle of such a nuanced thing as "good" with a tool like this. Likely info capture and user feedback would be a first port of call for a substantive change, internal testing is always ongoing, but such a low sample size.
I like the idea, but for me the displayed rankings were not particularly good, perhaps it needs a bit more data
Also I know that depending on the days / weeks / mood I will want to read different content from HN, so I guess there should still be like 30% of "random articles" in each category just to create some noise
The generated page was really off for me — I had read most of the posts it ranked, at least a little, and most recommended as skips were some of my favorite recent submissions, and vice versa with dives.
On the other hand I'm not sure I'd want to use something like this much as something I like about HN are the pleasant surprises. Maybe as a side page or something if I were really in a rush?
yeah seeing so many people using this its clear we should add some way for people to indicate when things felt off vs good, so that we can start tweaking the system, maybe with some evals.
We played around with the idea of a "fun" or "random" category, but ultimately didn't include it in this little first demo, as we found it super hard to have it not be just literally random (although that might not be a bad thing as you say)
On the topic of different moods and headspaces, thats one of the things more broadly we are really thinking about outside of this demo, and hadn't really considered for here but should. What different data can we use (in this case maybe just a different survey for different "profiles"), and how can a user manage those different profiles and front pages will be questions to answer.
Id be really interested to know if anyone has done topic grouped or themed frontpages for hackernews, as this would map well to that concept. ill have a look.
As far as rating posts: user favorites are public, and you could ask for a copy+paste of a few pages of upvoted stories if someone is not using the favorites feature. The stories that have been commented on are also a pretty strong public signal.
this is an angle we honestly didn't think about (we are pretty much long time lurkers) but accessing existing HN content is a great idea! I didn't even know there was a page of upvoted submissions :) It doesn't look like thats available via the API, but a copy paste of some text should work just aswel, all we pass through is titles and urls to the LLM anyway to generate the profile so its much the same.
More generally a next feature we want for ourselves is a way to add just some generic text and "update" the profile with that, rather than generate it fresh exclusively off of the 30 examples. This circles back to us using this as a focus point to think about what data is enough to generate a good user profile, and what good is.
Given the nature of the small pool(And the way they naturally exclude / includes topics), I'd strongly prefer if it had some way of adding more than 30 samples, maybe keep track of each set calibration taken and compare?
thanks for checking it out and taking the time to give some feedback!
When weve been testing things, we often find that if there wasnt a great match between the options when picking preferences, and whats currently on the front page, that the context it generates will result in a lot of skips (understandably, but not great UX). Right now can try regenerating your context (and going through the process again), or manually editing it to get to different results.
Theres also some work for us to better select the options when picking preferences, or ensuring we always surface some deep dives.
Applying the same process to more pages, or bubling up content from multiple pages, or new is a great idea. cool to hear thats where you would look.
It put this post at the top of my feed, which is cool, because it's incredibly relevant to my interests. I used to work on something similar, but way before LLMs were a thing.
Would you be willing to share some more of the architecture/tech stack?
yeah we see that a bit as well, i promise its not hardcoded in haha.
On the LLM side of things we are using Gemini 2.5 flash, mostly for speed, and found it to be reasonably good quality at a vibe level compared to something heavier like claude 4, probably because we've worked hard to keep the task very simple and explicit. But in saying that there are a bunch of comments on quality that really highlight that if we want to get serious about that we should put in some user feedback loops and evals.
Its all in JS/TS, using vercel ai for the LLM calls, storage is local, but in order to really dig into quality we might start saving things, but to do that well we'd have to add auth/users etc. and we wanted to keep it light for a demo. We have been recently exploring langfuse for tracing, and are really liking that, and will probably look at using them for first pass evals when we get to it for this project.
We also talked quite a bit about non-LLM recsys and aside from time to set up and do well, something I really like is the sense of transparency and agency. you can see your profile, and edit it if you like to see the change in your results. I almost think wed lean further into that rather than folding in some trad DS or recsys stuff even if that might make the results better. Just musings at this point though.
Thanks for the elaborate response! I recently looked at doing something similar and ended up with the option of using vector embeddings. Is that something you've considered?
For a first pass having some emb search would require we store and process a lot, which we dont do for this light weight demo. all the posts and recommendations are done on the fly, only looking at the current top page and re-ranking. as some others suggest, if we wanted to expand the recs more broadly than the front page then it might be a great way to run a first cut before running our rec llm on it.
For richer data to build a profile its something we look at a bunch for other projects, which could get folded in here if we decide to make it more persistent.
i found the "personal profile" that it created almost more interesting than the actual feed itself. from quite a small sample of posts it had mapped and summarised my interests really well.
i think the bit that needs the most work is classifying each post on the home page; quite a lot of posts that i would mark as "Dive" given its own classification of me ended up as "Skim".
Yes us to! I spend heaps of time playing with the link between preferences and the profile.
We aren't really sure yet how best to surface _why_ the model predicts what it does. You can hover over the skim label and there is a bit of reasoning text, which might shed some light on why for now. We will think more about how to make these relationships more clear in the process of tightening them up and generally improving them.
Once the relationships are a bit more clear theres probably the 80/20 rule of work to tighten up those predictions.
Are you referring to what's in the textarea on the edit profile text dialog? Seems to be a simple concatenation of the titles according to how I tagged them. If it's indeed this what you're referring to, what did you find interesting about this?
Mine has this analysis at the top, which I found interesting:
## Analysis of user's tech interest:
The user shows a strong interest in foundational computing concepts, historical perspectives on technology, and cutting-edge advancements in AI/ML, particularly those related to model architecture and efficiency. They are also drawn to low-level programming, system design, and hardware. Conversely, they seem less interested in business/startup narratives, general data manipulation tools, and consumer-oriented tech news unless it has a deep technical underpinning.
I love this. I’ve always considered doing something similar with a traditional recsys model.
The only feature I’d love to see, is there are many posts where I’m more interested in the HN comments, rather than the articles themselves. It would be great to see this incorporated somehow.
thats grand, thank you! we really want to look at both using comments as a source for preference, and applying your preferences to comments to perhaps flag/sort/filter once you are on a page. Will have to think about the user flow, and the way in which preferences differ for comments vs posts
It tried to figure out my interests based off my answers. Little does it know that I'm actually just interested in anything that has a catchy/funny title.
hahaha thats a great test case, gona remember that for evals. Did it notice anything? or did it just get confused. we found if you have random preferences it often just says to skim half and skim half, which i suspect it might do in your case.
It _in theory_ should try and pick up a content style (funny stuff??) even if the tech is seemingly random, but i wouldn't be surprised if it just failed.
Sadly not! It lives in our FE monorepo at work, it just kind of jumped out at us as something to split out and demo once we started using it to test some of our ideas. Its something wed consider for sure, but for now the lack of an open repo is kind of a bit of tech debt in a way. Easier to dev in our monorepo to get something out fast.
we came to this by looking at how a "user profile" in plain english could be both used and generated by LLMs, but once we were looking at this we did discuss traditional recsys. Two things against it for this usecase: bootstrapping preferences with a low number of data points, and no "unified" storage of all users preferences or pre-existing dataset is difficult with trad ML or statistical methods. Also having your preferences or "model" if you will in plain english gives a sense of agency, transparency and individuality to your recomendataions that are at least difficult, if not impossible to communicate with other types of models.
Id love to have those assumptions challenged though, if there are examples you could point me towards.
I genuinely find that interesting to hear. what about the 30 examples felt different to say the frontpage? (assuming it did)
On a meta level i was suuuuper conscious of writing every word of this post/comments myself, as my prior is that HN's community is very intollerant of and highly sensitive to low effort content, whether via AI or not. This is despite using AI tools for lots of other parts of work (drafting, coding, summarising, brainstorming etc).
Do you think HN has become more accepting of AI slop, the slop is becoming harder to detect, or isnt as discerning as i assume?
Somewhat tangential but a oddly under-explored side of the LLMs+HN data projects - HN search that does a good job finding HN submission/comment results for the type of things often asked in Ask HN that can be answered by HN searches. 'How do I learn about [some nice thing or another]', etc. People asking this sort of thing often don't know the right keywords so pure keyword search often doesn't work well but more latent-spacey things, in theory, could. Another related one is "can you LLMgenerate something akin to dang's 'Related' posts".
hackernews is such a rich source of "non trivial" content, and we are really seeing this when using it for a project compared with just consuming it as an individual.
We are focusing right now on how comments could be used to build up a better user context, and your comment has made me think about how we can feed comments in (instead of just titles and urls) for your selected preferences to make a better profile, without needing to scrape anything (expensive and slow).
But i think that would only work because of their quality and relevance generally, which would for sure make it an interesting knowledge source for pure LLM search. Feels like something someone should build or maybe already has! keen to see if anyone knows of projects like that.
Feels like something someone should build or maybe already has!
Yep, it very much feels like that but it doesn't seems to have happened yet. Even a not-entirely-quite-working yet attempt there could be an interesting thing/discussion.
This is a really cool idea. I love that you expose the personal profile as markdown. Reminds me of this article and exposing the system prompt: https://koomen.dev/essays/horseless-carriages/. Well done!
I wish I could paste my profile into a new device without needing to sit through the quiz again.
Thats a great point, and a good articl. I think the example they bring up as "good" and what we are leaning into here, is the idea of transparency and agency of being able to see and modify your profile.
The tension we have been finding is that we dont want to require people to "know how to prompt" to get value out of having a profile, hence our ongoing thinking around how to bootstrap good personal profiles from various data sources.
As Koomen notes, a good profile feels like it could be the best weapon against "AI slop" in a case I want something sharp and specific. But getting to that requires knowing how to prompt most of the time.
Makes sense. Bootstrapping the prompt based on some sample articles is smart.
Funny, I did the swipe thing and then the first result was this post with a [dive] tag. No idea if cheeky or if it actually got that from my choices, but I had a laugh anyway. Neat PoC!
edit: ooh, I see what the swiping did:
## Analysis of user's tech interest: The user demonstrates a strong interest in advanced technical topics, particularly in the realm of artificial intelligence, machine learning, and low-level systems programming/security (e.g., kernel exploitation). They are drawn to articles that involve practical application, model creation, and deep dives into complex technical architectures. Their interest in "Show HN" articles suggests an appreciation for new, innovative projects, especially those with a technical or AI focus. They show less interest in general hardware announcements (like new microcontrollers), historical tech accounts, or very niche, non-AI/ML/security-related programming topics.
Yeah, that's pretty much spot on. Wonder if there's a way to match that against the topics I actually commented on, but at a glance it's pretty cool!
Nice, great to see where it nails it, and has the right feel! Using your own comments directly, or in the first instance allowing users to paste in sections of their commented on or upvoted history is a great next step. Not sure yet whether the flow should be in place of or in addition to doing the survey.
I've been wanting this for a while! As an alpha prototype its great. Skip/Skim/Deep dive feels like the right breakdown to me. Having a different color but same UI feels right too.
Other than quality of life stuff (multiple pages for example), I'd like to see it continually learn.
A few things got miscategorized and I'd love for it to naturally correct that with additional input from me.
Hey so cool to hear! the skim/skip/dive mapped to how we use things, but thats sample size of 3 haha.
The idea of having some kind of thumbs up/down on what you see after getting recs, that gets added to your preferences, or being able to do another round of preferences (rather than just re-doing them like we have now) is for sure on our next steps if we continue with. Were not quite sure what the feedback loops will be yet (we did look at adding whole webhistory for example but that felt like a bit much and pretty invasive).
For the miscats, on a meta level what we are generally interested in is whether they come from compression of the preferences into your user profile (essentially if more or better data is the path to better context for such a specific usecase, or whether there is more bang for buck optimizing the various prompts. Keen to hear if its obvious from looking at your profile what was the case.
If we get serious with this evals are a must next step. We are only 2 days in at the moment :)
As it is currently written, this is less useful the more niche your interests are. I think for such users, looking at their comment history or upvoted history might be useful instead of or in addition to just sampling recent, popular articles.
In my case, none of the topics I most like to read about and discuss on HN (package management, software freedom, next-gen CLI tools, next-gen shells, philosophy, desktop Linux, functional programming, hacker history, literate programming, Emacs, bitching about common development practices, programming language design, configuration languages) managed to appear in the 30-post sample I used. The profile it wrote for me was pretty good considering that, but definitely not great.
The assessment was also mistaken about my degree of interest in "low level" technical details like binary file formats (in fact it's rather low, although it has gradually increased over time), and my degree of interest in theoretical computer science issues (in fact it's high, but all of the theoretical papers in the sample were about machine learning, which was not an area of academic focus for me).
I do really like the simplicity and customizability of this (exposing the profile as Markdown and making it editable is awesome), and the quality of the results is very good given the tiny input size. But if your primary interests are not super aligned with the mainstream on HN, you won't get a chance to demonstrate that you like them. If users could type a few terms to say what their biggest interests are before running through the samples, this could work even better for people like me.
It would also be interesting if this could work based on article contents and not just headlines. Sometimes I open something and close it immediately, or I open it undecided as to whether I will skim or read closely.
This is great to hear in such detial. One of the first cabs of the rank to improve this would be greater user control over what preferences to include, and/or smarter selection of the pool to select. This kind of focused preference is super important, especially as i use hackernews, and clearly for you and i suspect others as well.
In fact I would pose that I have a couple of disparate interests or "profiles" that i would like to have greater control over/support in generating, that are non overlapping sets of topics and types of content. The ability to have greater agency in creating them and managing them is something we are keen to explore.
The article comments one is a toughie, as LLM use skyrockets when you scrape and consume content from the links. It would be awesome to include it, but would likely need to be paid, just from a cost perspective.
Really appreciate the detail here, this makes it easier to turn your examples into a test/eval/feature case.
>I have a couple of disparate interests or "profiles" that i would like to have greater control over/support in generating, that are non overlapping sets of topics and types of content
This sounds like a great feature! My appetites for different clusters content certainly vary according to my mood! Perhaps "mood" would actually be a cute-but-clear name for such distinct/multiple profiles. :)
> The article comments one is a toughie, as LLM use skyrockets when you scrape and consume content from the links. It would be awesome to include it, but would likely need to be paid, just from a cost perspective.
Hm. That is a good (and in retrospect, obvious) point. If it makes the feed a lot better, I think it could certainly be worth it for some users. If it only makes a small difference, maybe not. It might be interesting for you to experiment and write about, since what kind of difference it will make isn't obvious (at least to me) up front.
yeah we are thinking a lot right now about good language for how to separate the notion of a "profile" as representative of who i am, from the concept of a profile that represents what mode im currently operating in right now. In the context of browsing, mood actually captures it quite well i agree. As we think wider to the different ways we might want a profile to help guide what content we receive or results we get when using other tools, that language isnt quite so clear yet.
We will have to do some combo of much more internal testing, construct evals, or just capture more info about peoples usage coupled with an ability to provide feedback in order to even get a handle of such a nuanced thing as "good" with a tool like this. Likely info capture and user feedback would be a first port of call for a substantive change, internal testing is always ongoing, but such a low sample size.
I like the idea, but for me the displayed rankings were not particularly good, perhaps it needs a bit more data
Also I know that depending on the days / weeks / mood I will want to read different content from HN, so I guess there should still be like 30% of "random articles" in each category just to create some noise
I felt similarly.
The generated page was really off for me — I had read most of the posts it ranked, at least a little, and most recommended as skips were some of my favorite recent submissions, and vice versa with dives.
On the other hand I'm not sure I'd want to use something like this much as something I like about HN are the pleasant surprises. Maybe as a side page or something if I were really in a rush?
yeah seeing so many people using this its clear we should add some way for people to indicate when things felt off vs good, so that we can start tweaking the system, maybe with some evals.
We played around with the idea of a "fun" or "random" category, but ultimately didn't include it in this little first demo, as we found it super hard to have it not be just literally random (although that might not be a bad thing as you say)
On the topic of different moods and headspaces, thats one of the things more broadly we are really thinking about outside of this demo, and hadn't really considered for here but should. What different data can we use (in this case maybe just a different survey for different "profiles"), and how can a user manage those different profiles and front pages will be questions to answer.
Id be really interested to know if anyone has done topic grouped or themed frontpages for hackernews, as this would map well to that concept. ill have a look.
As far as rating posts: user favorites are public, and you could ask for a copy+paste of a few pages of upvoted stories if someone is not using the favorites feature. The stories that have been commented on are also a pretty strong public signal.
this is an angle we honestly didn't think about (we are pretty much long time lurkers) but accessing existing HN content is a great idea! I didn't even know there was a page of upvoted submissions :) It doesn't look like thats available via the API, but a copy paste of some text should work just aswel, all we pass through is titles and urls to the LLM anyway to generate the profile so its much the same.
More generally a next feature we want for ourselves is a way to add just some generic text and "update" the profile with that, rather than generate it fresh exclusively off of the 30 examples. This circles back to us using this as a focus point to think about what data is enough to generate a good user profile, and what good is.
Given the nature of the small pool(And the way they naturally exclude / includes topics), I'd strongly prefer if it had some way of adding more than 30 samples, maybe keep track of each set calibration taken and compare?
Checked it out and it's offering 2 skim, and the rest are skip?
I had an expectation that it'd go through posts and give me stuff i'd be interested in. Like here's 25 posts that would be interesting?
Only frontpage? no second page? No sort by new, which is my preferred.
thanks for checking it out and taking the time to give some feedback!
When weve been testing things, we often find that if there wasnt a great match between the options when picking preferences, and whats currently on the front page, that the context it generates will result in a lot of skips (understandably, but not great UX). Right now can try regenerating your context (and going through the process again), or manually editing it to get to different results.
Theres also some work for us to better select the options when picking preferences, or ensuring we always surface some deep dives.
Applying the same process to more pages, or bubling up content from multiple pages, or new is a great idea. cool to hear thats where you would look.
[dead]
It put this post at the top of my feed, which is cool, because it's incredibly relevant to my interests. I used to work on something similar, but way before LLMs were a thing.
Would you be willing to share some more of the architecture/tech stack?
yeah we see that a bit as well, i promise its not hardcoded in haha.
On the LLM side of things we are using Gemini 2.5 flash, mostly for speed, and found it to be reasonably good quality at a vibe level compared to something heavier like claude 4, probably because we've worked hard to keep the task very simple and explicit. But in saying that there are a bunch of comments on quality that really highlight that if we want to get serious about that we should put in some user feedback loops and evals.
Its all in JS/TS, using vercel ai for the LLM calls, storage is local, but in order to really dig into quality we might start saving things, but to do that well we'd have to add auth/users etc. and we wanted to keep it light for a demo. We have been recently exploring langfuse for tracing, and are really liking that, and will probably look at using them for first pass evals when we get to it for this project.
We also talked quite a bit about non-LLM recsys and aside from time to set up and do well, something I really like is the sense of transparency and agency. you can see your profile, and edit it if you like to see the change in your results. I almost think wed lean further into that rather than folding in some trad DS or recsys stuff even if that might make the results better. Just musings at this point though.
Thanks for the elaborate response! I recently looked at doing something similar and ended up with the option of using vector embeddings. Is that something you've considered?
For a first pass having some emb search would require we store and process a lot, which we dont do for this light weight demo. all the posts and recommendations are done on the fly, only looking at the current top page and re-ranking. as some others suggest, if we wanted to expand the recs more broadly than the front page then it might be a great way to run a first cut before running our rec llm on it.
For richer data to build a profile its something we look at a bunch for other projects, which could get folded in here if we decide to make it more persistent.
This looks great! At first glance the dive/skim/skip suggestions it offered for me are well judged (I'm now actually diving into the dive ones).
During the rating part, am I supposed to be clicking on the link to decide how to rate it? Or should I be basing myself only on the title?
how ever you like! i tend to do it off the title, but you could click into it if you like.
i found the "personal profile" that it created almost more interesting than the actual feed itself. from quite a small sample of posts it had mapped and summarised my interests really well.
i think the bit that needs the most work is classifying each post on the home page; quite a lot of posts that i would mark as "Dive" given its own classification of me ended up as "Skim".
Yes us to! I spend heaps of time playing with the link between preferences and the profile.
We aren't really sure yet how best to surface _why_ the model predicts what it does. You can hover over the skim label and there is a bit of reasoning text, which might shed some light on why for now. We will think more about how to make these relationships more clear in the process of tightening them up and generally improving them.
Once the relationships are a bit more clear theres probably the 80/20 rule of work to tighten up those predictions.
Are you referring to what's in the textarea on the edit profile text dialog? Seems to be a simple concatenation of the titles according to how I tagged them. If it's indeed this what you're referring to, what did you find interesting about this?
Mine has this analysis at the top, which I found interesting:
## Analysis of user's tech interest: The user shows a strong interest in foundational computing concepts, historical perspectives on technology, and cutting-edge advancements in AI/ML, particularly those related to model architecture and efficiency. They are also drawn to low-level programming, system design, and hardware. Conversely, they seem less interested in business/startup narratives, general data manipulation tools, and consumer-oriented tech news unless it has a deep technical underpinning.
Very interesting, but like others suggested I'd like for it to use my upvoted submissions and comments to build a profile about me.
I love this. I’ve always considered doing something similar with a traditional recsys model.
The only feature I’d love to see, is there are many posts where I’m more interested in the HN comments, rather than the articles themselves. It would be great to see this incorporated somehow.
Awesome work though. Will bookmark!
thats grand, thank you! we really want to look at both using comments as a source for preference, and applying your preferences to comments to perhaps flag/sort/filter once you are on a page. Will have to think about the user flow, and the way in which preferences differ for comments vs posts
It tried to figure out my interests based off my answers. Little does it know that I'm actually just interested in anything that has a catchy/funny title.
hahaha thats a great test case, gona remember that for evals. Did it notice anything? or did it just get confused. we found if you have random preferences it often just says to skim half and skim half, which i suspect it might do in your case.
It _in theory_ should try and pick up a content style (funny stuff??) even if the tech is seemingly random, but i wouldn't be surprised if it just failed.
Nifty! Is there a github or source for this?
Sadly not! It lives in our FE monorepo at work, it just kind of jumped out at us as something to split out and demo once we started using it to test some of our ideas. Its something wed consider for sure, but for now the lack of an open repo is kind of a bit of tech debt in a way. Easier to dev in our monorepo to get something out fast.
Did you consider using more “traditional” recommendation systems? (and maybe using LLMs to create synthetic preferences…)
we came to this by looking at how a "user profile" in plain english could be both used and generated by LLMs, but once we were looking at this we did discuss traditional recsys. Two things against it for this usecase: bootstrapping preferences with a low number of data points, and no "unified" storage of all users preferences or pre-existing dataset is difficult with trad ML or statistical methods. Also having your preferences or "model" if you will in plain english gives a sense of agency, transparency and individuality to your recomendataions that are at least difficult, if not impossible to communicate with other types of models.
Id love to have those assumptions challenged though, if there are examples you could point me towards.
Having to rate the 30 examples made me realise just how much HN is dominated by LLM content these days. Kinda sad.
I genuinely find that interesting to hear. what about the 30 examples felt different to say the frontpage? (assuming it did)
On a meta level i was suuuuper conscious of writing every word of this post/comments myself, as my prior is that HN's community is very intollerant of and highly sensitive to low effort content, whether via AI or not. This is despite using AI tools for lots of other parts of work (drafting, coding, summarising, brainstorming etc).
Do you think HN has become more accepting of AI slop, the slop is becoming harder to detect, or isnt as discerning as i assume?
I'm not talking about the content but about the topics.
Somewhat tangential but a oddly under-explored side of the LLMs+HN data projects - HN search that does a good job finding HN submission/comment results for the type of things often asked in Ask HN that can be answered by HN searches. 'How do I learn about [some nice thing or another]', etc. People asking this sort of thing often don't know the right keywords so pure keyword search often doesn't work well but more latent-spacey things, in theory, could. Another related one is "can you LLMgenerate something akin to dang's 'Related' posts".
hackernews is such a rich source of "non trivial" content, and we are really seeing this when using it for a project compared with just consuming it as an individual.
We are focusing right now on how comments could be used to build up a better user context, and your comment has made me think about how we can feed comments in (instead of just titles and urls) for your selected preferences to make a better profile, without needing to scrape anything (expensive and slow).
But i think that would only work because of their quality and relevance generally, which would for sure make it an interesting knowledge source for pure LLM search. Feels like something someone should build or maybe already has! keen to see if anyone knows of projects like that.
Feels like something someone should build or maybe already has!
Yep, it very much feels like that but it doesn't seems to have happened yet. Even a not-entirely-quite-working yet attempt there could be an interesting thing/discussion.
I was able to find this post, which is a little old in LLM land, but appears to be pretty functional! https://news.ycombinator.com/item?id=40238509
The post and comments include a bunch of other tools that feel similar, and the tool itself works.
Ill have to take some time to use it, but also see what i can learn about how they've consumed and used HN comments more generally.
If only there was some way to find all the similar related previous submissions (there have been a few :)
[dead]