It's a really interesting project. But boy do they make it hard to participate.
* Article doesn't provide a direct link to the topic mission
* Signup is pretty easy. Well organized and even gently requires you to have two forms of 2FA.
* Sign up complete. Go back to the primary page and try to find the mission. A little buried but not too deep.
* Notice I'm not signed in. Ok, let's do that. Now I'm back on the main page and navigate back. Find the first document and open it. Really interesting to scan through the doc and to read. People back then generally had really nice handwriting.
* Ok, what next, how do I transcribe? ... ? Oh it says I'm not logged in again. Fine, click the link and...
* I'm logged in and directed back to the main page, again.
Look, this is an interesting project and I'd love to spend my spare cycles to help out. But they really need to clean up this process.
Volunteers shouldn't have to jump through kinda poorly designed interfaces to help out.
The social post embedded in the page links directly to this page with all the instructions. Once I created an account and signed in I just selected a state in the original tab and was right there and could start translating.
Do you perhaps have uBlock Origin enabled or some other limitation on Javascript/cookies that might be messing with your login status?
Before commenting asking about why they don't just use LLMs, please note that the article specifically calls out that they do, but it's not always a viable solution:
> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.
The document at the top is likely an especially easy document to read precisely because it's meant to be the hook to get people to sign up and get started. It isn't going to be representative of the full breadth of documents that the National Archives want people to go through.
OK, fair enough, but can you find one in this article that's hard for an LLM? The gnarliest one I saw, 4o handled instantly, and I went back and looked carefully at the image and the text and I'm sold.
Like if this is a crowdsourcing project, why not do a first pass with an LLM and present users with both the image and the best-effort LLM pass?
Later
I signed up, went to the current missions, and they all seem to post post-1900 and all typeset. They're blurry, but 4o cuts through them like a hot knife through butter.
> Like if this is a crowdsourcing project, why not do a first pass with an LLM and present users with both the image and the best-effort LLM pass?
Possibly for the reason that came up in your other post: you mentioned that you spot checked the result.
Back when I was in historical research, and occasionally involved in transcription projects, the standard was 2-3 independent transcriptions per document.
Maybe the National Archive will pass documents to an LLM and use the output as 1 of their 2-3 transcriptions. It could reduce how many duplicate transcriptions are done by humans. But I'll be surprised if they jump to accepting spot checked LLM output anytime soon.
My parents have saved letters from their parents which are written in cursive but in two perpendicular layers. Meaning the writing goes horizontally in rows and then when they got to the end of the page it was turned 90 degrees and continued right on top of what was already there for the whole page. This was apparently to save paper and postage. It looks like an unintelligible jumble but my mother can actually decipher it. Maybe that’s what the LLMs are having trouble with?
Are they having trouble? You can sign up right now and get tasks from the archive that seem trivial for 4o (by which I mean: feed a screenshot to 4o, get a transcription, and spot check it).
My guess is because it’s the Smithsonian, they’re just not willing to trust an LLM’s transcription enough to put their name on it. I imagine they’re rather conservative. And maybe some AI-skeptic protectionist sentiments from the professional archivists. Seems like it could change with time though.
> My guess is because it’s the Smithsonian, they’re just not willing to trust an LLM’s transcription enough to put their name on it. I imagine they’re rather conservative
I expect thats a common theme from companies like that, yet I don't think they understand the issue they think they have there.
Why not have the LLMs do as much work as possible and have humans review and put
their own name on it? Do you think they need to just trust and publish the output of the LLM wholeheartedly?
I think too many people saw what a few idiot lawyers did last year and closed the book on LLM usage.
One that require additional work beyond simply feeding the image into the model would be this example which is a mix of barely legible hand written cursive and easy to read typed form. [0] Initially 4o just transcribes (successfully) the bottom half of the text and has to be prompted to attempt the top half at which point it seems to at best summarize the text instead of giving a direct transcription. [1] In fact it seems to mix up some portions of the latter half of the typed text with the written text in the portion of it's "transcription" about "reduced and indigent circumstances".
[1] Reproducing here since I cannot share the chat since it has user uploaded images. "
The text in the top half of the image is handwritten and partially difficult to read due to its cursive style and some smudging. Here's my best transcription attempt for the top section:
...resident within four? years, swears and says that the name of the John Hopper mentioned in the foregoing declaration is the same person, and he verily believes the facts as stated in the declaration are true.
He further swears that the said John Hopper is in reduced and indigent circumstances and requires the aid of his country.
The declarant further swears he has no evidence now in his power of service, except the statement of Capt. (illegible name), as to his reduced circumstances ...
Sworn to before me, this day...
Some parts remain unclear due to the handwriting, but let me know if you'd like me to attempt further clarification on specific sections!"
I'm doing some genealogy work right now on my family's old papers covering the time period from recent years back to the late 17th century. Handwriting styles changed a lot over the centuries and individuals can definitely be identified by their personal cursive style of writing and you can see their handwriting change as they aged.
Then you have the problem that some of these ancestors not only had terrible penmanship but also spelled multi-syllabic words phonetically since they likely were barely educated kids who spent more time when they were young working on the farm or ranch instead of attending school where they would've learned how to spell correctly.
I don't know whether your LLM can handle English words spelled phonetically written in cursive by an individual who had no consistency in forming letters in the words. It is clear after reading a lot of correspondence from this person that they ignored things that didn't seem important in the moment like dotting i's or crossing t's or forming tails on g's, p's, j's, or even beginning letters consistently since they switched between cursive and block letters within a sentence, maybe while they paused to clarify their thoughts. I don't know but it is fascinating to take a walk through life with someone you'll never meet and to discover that many of the things that seemed awesome to you as a kid were also awesome to them and that their life had so many challenges that our generations will never need to endure.
Some of my people have the most beautiful flowing cursive handwriting that looks like the cursive that I was taught in grade school. Others have the most beautiful flowing cursive with custom flourishes and adornments that make their handwriting instantly recognizable and easy to read once you understand their style.
I think there are plenty of edge cases where LLMs will take a drunkard's walk through the scribble and spit out gibberish.
I'm reminded of an old joke though.
Ronald Reagan woke up one snowy Washington, DC morning and took a look out of the window to admire the new-fallen snow. He enjoys the beautiful scene laid out before him until he sees tracks in the snow below his window and a message obviously written in piss that said - "Reagan sucks".
He dispatched the Secret Service to the site where samples were taken of the affected snow and photos of the tracks of two people were made.
After an investigation he receives a call from the Secret Service agent in charge who tells him he has some good news and some bad news for him.
The good news is that they know who pissed the message. It was George HW Bush, his Vice President. The bad news is that it was Nancy's handwriting.
Some are significantly harder to read. I took the page below and tried to get GPT 4o to transcribe it and it basically couldn't do it. I'm not going to sit and prompt hack for ages to see if it can but it seems unable to tackle the handwritten text at the top. When I first just fed it the image and asked for a transcription it only (but successfully) read the bottom portion, prompted for a transcription of the top it dropped into more of a summary of the whole document mainly pulling some phrases from the bottom text. (Sadly can't share it but I copied it's reply out in a comment upthread) [0]
It was more successful at a few others I tried but it's still a task that requires manual processing like a lot of LLM output to check for accuracy and prompt modification to get it to output what you need for some documents.
Ok I did one letter, from a woman in 1814 writing to James Monroe (then Secretary of State) asking for a passport to go to Scotland to get her late brother's property. What a trip! So enjoyable to get into the flow once you've "synchronized" with the persons handwriting. Furthermore, due to the fact that you're reading and re-writing word for word of whatever you're transcribing, the stories you end up reading have tremendous memory-stick. This is not surprising, considering that you are dedicating an inordinate amount of time per page, but it's a welcome side effect when you try and recollect.
> Furthermore, due to the fact that you're reading and re-writing word for word of whatever you're transcribing, the stories you end up reading have tremendous memory-stick. This is not surprising, considering that you are dedicating an inordinate amount of time per page, but it's a welcome side effect when you try and recollect.
This was something I enjoyed when I decided to learn a language by translating short stories. (Edit: Of course, you have to choose an author whose diction you respect. Your unfamiliarity with the target language encourages you to mull over the author's use of diction and the nuances the author is trying to convey, and then find appropriate diction in English. This means you spend a long time immersed in the imagery.)
cheers! I was looking for something semi productive to sink a Friday night into
on a more serious note, working through a transcription project for letters and journals that nobody has touched since they've been archived is such a wonderful feeling. Aside from being in front of the physical document itself, your degree of separation from the writer and point is time is vanishingly small!
I always like to observe when they cross something out or make a mistake and think about what could have caused that. Did a friend pass by the door and scare them? Did they get distracted looking out the window? It's all so close and yet so far away :)
I’m interested to give this a go because I want to practice reading cursive. I do a lot of longhand writing including writing all my notes in cursive. It’s exciting to watch my binding fill up with all sorts of different subjects!
I like to write in cursive for a few reasons
1. I find it makes my hand cramp less
2. It offers some shallow privacy in public
3. I don’t want to lose the skill
4. It’s fun!
All of the same reasons I love practicing a little calligraphy! I love how it looks as well. I don’t use a special pen but just add my own style to my cursive to make it look even nicer. But I used to write my notes in school with calligraphy (mostly because it gave me an excuse to not care about the subject) but it made the teachers hate me because I would never finish copying their scribbles fast enough.
Seems like something that some of those big AI companies that are desperately starved of training material could chip in on, no? Actually do something for the public good, spend a few cents of that VC money, get some high-quality training data out of it?
This reminded me of something the historian Megan Marshall wrote in the introduction to her book The Peabody Sisters: Three Women Who Ignited American Romanticism (2005):
“I became expert in deciphering the sisters’ handwriting, and that of their ancestors, parents, and friends. Each era and each correspondent presented different challenges. Some hands were sprawling, some spindly, some cramped; t’s went uncrossed at the ends of words, and f’s and s’s were interchanged; spelling, capitalization, and punctuation could be erratic or idiosyncratic. Often, to save paper and postage, the sisters turned a single sheet ninety degrees and wrote back across a page already covered with handwriting. I learned to be especially attentive to these cross-written lines, in which the sisters invariably confided their deepest feelings in the last hurried moments of closing a letter. Here I would find the urgent personal message that had been put off for the sake of dispensing news or settling business. In one such postscript, I discovered Elizabeth’s account of a conversation with Horace Mann in which the two spoke frankly of their love for each other and finally settled on what it meant.”
A photograph of a letter with cross-writing is here:
This is all very cool so I’m not trying to be dismissive. In a lot of ways, giving a hobby out as a way to participate in the national archives is an end in itself.
But…computers can definitely do this way better, right?
I had the same thought but maybe on old hand writing they can't?
EDIT:
I tried giving the sample to 4o and it gave:
The following is the declaration of James Lambert, a soldier of the Revolutionary War in North America.
The said James Lambert this day personally appeared in the Probate Court of the County of Dearborn in the State of Indiana and at the November Term of said Court (1841), it being a court of record created by the laws of Indiana and made oath that:
On the 25th day of March 1842, he will be eighty-five years old, that he was born in the State of Maryland, that he is now a resident of said county and has been for the 27 years last past; that he has lived in Virginia, Maryland, and Pennsylvania...
> This is all very cool so I’m not trying to be dismissive. In a lot of ways, giving a hobby out as a way to participate in the national archives is an end in itself.
> But…computers can definitely do this way better, right?
No.
Cursive writing is analog and fluid, lacking consistency across authors and often inconsistent by an individual author as well. When done well, it could be classified as its own art form. When done poorly, it can resemble the path walked by a chicken on meth.
The handwriting in some of these snippets, while sometimes difficult to read for one reason or another, is nonetheless beautiful: did everyone who wrote have such great handwriting back then?
I'm looking at the piece in the Instagram post linked by the page, which begins, "honor of holding in their service". The lines are so straight, the letters are so uniform!
As someone with terrible handwriting but decent cursive, i think cursive provides a better structure for achieving cleaner penmanship compared to non-cursive writing. My theory is that cursive’s consistency of soft, flowing loops rather than a mix of abrupt angles and disconnected lines helps create a more uniform result.
I also remember teachers telling you when writing cursive to seldom lift your hand from the page. I think that act of keeping your pen on the page for most of the writing process encourages a smoother and more natural flow, reducing the chance of jerky, uneven strokes
“The following is the declaration of James Lambert, a soldier of the Revolutionary War in North America.”“The said James Lambert, on this day personally appeared in the Probate Court of the County of Dearborn in the State of Indiana, at the November Term of said Court [1841], it being a court of record created by the laws of Indiana, and made oath
that on the 25th day of March 1842 he will be eighty‐five years old; that he was born in the State of Maryland; that he is now a resident of [said] county and has been for the [27] years last past; that he has lived in Virginia, Maryland, [and Pennsylvania]; that…”
These kinds of problems, matching up cursive to actual text, would seem to play to the absolute best strengths of an LLM, given how much basic language structure the models encode.
> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.
I've seen people do that, and the results are.. just sad. These modern models insert their twitter-era "what grabs attention must be true" view into the very little authentic past we still possess.
Funnily enough, there have been a few times over the past couple of years I've been asked by younger co-workers to read something for them that was written in cursive. I hadn't really realized it had become such a (comparatively) rare skill. This fact is making me feel older than my actual 50th birthday did!
Not that I can tell, unless you encounter a teacher who (personally) believes it’s worthwhile.
The real problem, IMO, is that they don’t teach cursive but also don’t teach typing. They’ve thrown laptops at the kids without giving them the basic skill necessary to be effective in that medium.
They stopped teaching cursive for a number of years but all the schools in my area start it around age 6 or 7 now. They start typing the next year with some horribly boring typing program.
They started teaching it again because it correlated with better outcomes for things seemingly unrelated to writing. And it was important to learn it before typing supposedly. There is probably some better way to accomplish whatever it is actually doing, but they don't seem to know that.
It might be nice for people to be able to actually read the documents in the National Archives rather than relying on a transcription or a mobile app.
I wonder if they've considered making a simple tutorial on how to read cursive? It's not that hard if you can already read printed English. And of course you can practice on documents in the National Archives.
It's exciting and fun to learn to read an unfamiliar script, like the runes on the cover of The Hobbit ... or the engraving-style cursive of the US Constitution.
I think it likely that reading the great variety of cursive styles makes simple teaching rather complicated. Folks who spent years in school reading and writing in cursive can quickly adapt to the various styles, in a way that I'm not sure it could be done in a simple tutorial.
Those two statements aren't at odds with each other.
For example, there's a great abundance of resources to learn about music theory and such too, the average person doesn't know such things because they aren't interested.
no, it says the opposite, that there is growing interest in bringing it back into curriculums in various states. but that's aside from the point that the smithsonian making a tutorial on reading cursive would just represent an additional resource, of which we are not lacking, to learn. whether or not we teach it is different, but finding a resource to learn is not hard.
I find the article's conflation of two topics involving cursive writing ignorant or disingenuous to the point that I almost wanted to respond with my own comment on that itself. If you study cursive writing in class, you are likely to learn simple and standard letterforms like Palmer script.
But the task requested by the National Archives is more akin to paleography where you can expect each author or work to have their own (region-based/family-based) handwriting that requires decipherment, even for experts. You may have encountered a coworker or schoolmate's indecipherable chicken scratch print writing; that is what you should expect, only cursive.
Actually I think in 2025 you are correct, we just haven’t got the best tech into the OCR software that’s out there in the real world. I just pasted the letter from the article into ChatGPT (4o) and asked “what does this old letter say?” The response:
—-
The following is the declaration of James Lambert, a soldier of the Revolutionary War in North America.
The said James Lambert on this day personally appeared in the Probate Court of the County of Dearborn in the State of Indiana and at the November Term of said Court (1841), it being a court of record established by the laws of Indiana and made oath that:
On the 25th day of March 1842 he will be eighty-five years old; that he was born in the State of Maryland; that he is now a resident of said county and has been for the 27 years last past; that he has lived in Virginia, Maryland, Pennsylvania…
I've been trying every state of the art OCR solution on my students' handwritten essays for fifteen years and have yet to find anything even close to acceptable.
No. Sign up and look at the current missions. A lot of what they want transcribed is totally straightforward to OCR --- not even LLM, OCR. Whatever's going on, and I'm not second-guessing them, a pretty big chunk of their problem appears to be well within the state of the art. The appeal to authority isn't going to play here, because you can just click through to the archives and see what they're trying to figure out.
> No. Sign up and look at the current missions. A lot of what they want transcribed is totally straightforward to OCR --- not even LLM, OCR. Whatever's going on, and I'm not second-guessing them, a pretty big chunk of their problem appears to be well within the state of the art.
If it's that easy, then do it and be the hero they want.
Or maybe, just maybe, "a pretty big chunk of their problem appears to be well within the state of the art" is a sweeping generalization lacking understanding of the difficulties involved.
There are two claims. The main one is that all of these documents are easy to individually transcribe by machine. The other is that a whole lot can be OCR'd, which is pretty simple to check.
That's not a claim that processing the entire archive would be trivial. And even if it was, whether that would make someone the "hero they want" is part of what's being called into question.
So your silly demand going unmet proves nothing.
Also, "give me an example please" is not a strawman!
If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description.
> So your silly demand going unmet proves nothing.
I made demands of no one.
> Also, "give me an example please" is not a strawman!
My identification of the strawman was that it referenced "find something hard" when I had said "be the hero they want" and that what is needed in this specific problem domain may be more difficult than what a generalization addresses.
> If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description.
Maybe this is the proof you demand.
LLM's are statistical prediction algorithms. As such, they are nondeterministic and, therefore, provide no guarantees as to the correctness of their output.
The National Archives have specific artifacts requiring precise textual data extraction.
Use of nondeterministic tools known to produce provably incorrect results eliminate their applicability in this workflow due to all of their output requiring human review. This is an unnecessary step and can be eliminated by the human reading the original text themself.
Whatever you want to call "If it's that easy, then do it"
> LLM's [...] Does that satisfy your demand?
That's a different argument from the one above where you were trying to contradict tptacek. And that argument is flawed itself. In particular, humans don't have guarantees either.
> provably incorrect results
This gets back to the actual request from earlier, which is showing an example where the machine performs below some human standard. Just pointing out that LLMs make mistakes is not enough proof of incorrectness in this specific use case.
I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with. I'm sorry, but I declare the burden of proof here to be switched. Can you find a hard one?
(I don't think you need to Wikipedia-cite "straw man" on HN).
> I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with.
Awesome.
Can you guarantee its results are completely accurate every time, with every document, and need no human review?
> I'm sorry, but I declare the burden of proof here to be switched.
If you are referencing my stating:
If it's that easy, then do it and be the hero they want.
Then I don't really know how to respond. Otherwise, if you are referencing my statement:
> Perhaps "random humans" can perform tasks which could reshape your belief:
>> OCR is VERY good
To which I again ask, can you guarantee the correctness of OCR results will exceed what "random humans" can generally provide? What about "non-random motivated humans"?
My point is that automated approaches to tasks such as what the National Archives have outlined here almost always require human review/approval, as accuracy is paramount.
> (I don't think you need to Wikipedia-cite "straw man" on HN).
I do so for two purposes. First, if I misuse a cited term someone here will quickly correct me. Second, there is always a probability of someone new here which is unaware of the cited term(s).
> > If it's that easy, then do it and be the hero they want.
> Then I don't really know how to respond.
If someone says a thing is easy, and you respond by demanding they do it a million times to prove that it's easy, you are the one that has screwed up the burden of proof.
Can I ask, did you sign up and look at what they're actually looking for? Show of good faith: can you give 3 of the headers for the top-level "missions" they have for transcriptions?
I would challenge you to find a picture of text that you think a human can read and OCR cannot. I’m happy to demonstrate. The text shown in this article is trivial.
The archivists themselves say that they run into such texts often enough that this program was needed:
> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.
They are absolutely aware of the advances in these tools, so if they say they're not completely there yet I believe them. One likely reason is that the models probably have less 1800s-era cursive in their training set than they do modern cursive.
It's likely that with more human-tagged data they could improve on the state of the art for OCR, but it's pretty arrogant to doubt the agency in charge of this sort of thing when they say the tech isn't there yet.
I don't necessarily agree with her conclusion because she wasn't participating directly in the thread and wasn't completely responsive to some of the points raised, but still, it appears that there are a few instances of difficult-to-read handwriting where OCR is still coming in second to skilled human interpretation.
Text that is _intentionally constructed_ to fool computers but not humans is obviously out of scope. But they’re generally easily solved with OCR these days anyway.
Not being rude was also an option, one you chose not to take for some reason. Seriously, all it would've taken was for you to say something like "there have been a lot of advancements so it's probably different than you remember". This conversation would've gone much smoother for you if you had.
And BugsJustFindMe can't downvote you, because it was a reply to him. So don't bite his head off over it. You got downvoted because you were a jerk, plain and simple.
Refraining from reflexively pooh-poohing AI with uninformed and/or out-of-date opinions is also an option, but not one often exercised on HN.
It gets old not being able to carry on a discussion without squinting at grayed-out text, simply because someone pointed out that humans aren't robots and should no longer have to emulate them.
I'm not too sure about that reading, I got "The following is the declaration of James Lambert a soldier of the Revolutionary War in South America." rather different
Hmm, interesting: “North America” does make sense, and 4o also seems to transcribe it that way, but the handwriting looks like it says “South America” to me.
It's a really interesting project. But boy do they make it hard to participate.
* Article doesn't provide a direct link to the topic mission
* Signup is pretty easy. Well organized and even gently requires you to have two forms of 2FA.
* Sign up complete. Go back to the primary page and try to find the mission. A little buried but not too deep.
* Notice I'm not signed in. Ok, let's do that. Now I'm back on the main page and navigate back. Find the first document and open it. Really interesting to scan through the doc and to read. People back then generally had really nice handwriting.
* Ok, what next, how do I transcribe? ... ? Oh it says I'm not logged in again. Fine, click the link and...
* I'm logged in and directed back to the main page, again.
Look, this is an interesting project and I'd love to spend my spare cycles to help out. But they really need to clean up this process.
Volunteers shouldn't have to jump through kinda poorly designed interfaces to help out.
The social post embedded in the page links directly to this page with all the instructions. Once I created an account and signed in I just selected a state in the original tab and was right there and could start translating.
Do you perhaps have uBlock Origin enabled or some other limitation on Javascript/cookies that might be messing with your login status?
The direct link to the mission that was in the social post. https://www.archives.gov/citizen-archivist/missions/revoluti...
Before commenting asking about why they don't just use LLMs, please note that the article specifically calls out that they do, but it's not always a viable solution:
> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.
The document at the top is likely an especially easy document to read precisely because it's meant to be the hook to get people to sign up and get started. It isn't going to be representative of the full breadth of documents that the National Archives want people to go through.
OK, fair enough, but can you find one in this article that's hard for an LLM? The gnarliest one I saw, 4o handled instantly, and I went back and looked carefully at the image and the text and I'm sold.
Like if this is a crowdsourcing project, why not do a first pass with an LLM and present users with both the image and the best-effort LLM pass?
Later
I signed up, went to the current missions, and they all seem to post post-1900 and all typeset. They're blurry, but 4o cuts through them like a hot knife through butter.
> Like if this is a crowdsourcing project, why not do a first pass with an LLM and present users with both the image and the best-effort LLM pass?
Possibly for the reason that came up in your other post: you mentioned that you spot checked the result.
Back when I was in historical research, and occasionally involved in transcription projects, the standard was 2-3 independent transcriptions per document.
Maybe the National Archive will pass documents to an LLM and use the output as 1 of their 2-3 transcriptions. It could reduce how many duplicate transcriptions are done by humans. But I'll be surprised if they jump to accepting spot checked LLM output anytime soon.
My parents have saved letters from their parents which are written in cursive but in two perpendicular layers. Meaning the writing goes horizontally in rows and then when they got to the end of the page it was turned 90 degrees and continued right on top of what was already there for the whole page. This was apparently to save paper and postage. It looks like an unintelligible jumble but my mother can actually decipher it. Maybe that’s what the LLMs are having trouble with?
Edit: apparently it’s called cross writing [1]
1: https://highshrink.com/2018/01/02/criss-cross-letters/
Are they having trouble? You can sign up right now and get tasks from the archive that seem trivial for 4o (by which I mean: feed a screenshot to 4o, get a transcription, and spot check it).
My guess is because it’s the Smithsonian, they’re just not willing to trust an LLM’s transcription enough to put their name on it. I imagine they’re rather conservative. And maybe some AI-skeptic protectionist sentiments from the professional archivists. Seems like it could change with time though.
> My guess is because it’s the Smithsonian, they’re just not willing to trust an LLM’s transcription enough to put their name on it. I imagine they’re rather conservative
I expect thats a common theme from companies like that, yet I don't think they understand the issue they think they have there.
Why not have the LLMs do as much work as possible and have humans review and put their own name on it? Do you think they need to just trust and publish the output of the LLM wholeheartedly?
I think too many people saw what a few idiot lawyers did last year and closed the book on LLM usage.
One that require additional work beyond simply feeding the image into the model would be this example which is a mix of barely legible hand written cursive and easy to read typed form. [0] Initially 4o just transcribes (successfully) the bottom half of the text and has to be prompted to attempt the top half at which point it seems to at best summarize the text instead of giving a direct transcription. [1] In fact it seems to mix up some portions of the latter half of the typed text with the written text in the portion of it's "transcription" about "reduced and indigent circumstances".
[0] https://catalog.archives.gov/id/54921817?objectPage=8&object...
[1] Reproducing here since I cannot share the chat since it has user uploaded images. " The text in the top half of the image is handwritten and partially difficult to read due to its cursive style and some smudging. Here's my best transcription attempt for the top section:
...resident within four? years, swears and says that the name of the John Hopper mentioned in the foregoing declaration is the same person, and he verily believes the facts as stated in the declaration are true.
He further swears that the said John Hopper is in reduced and indigent circumstances and requires the aid of his country.
The declarant further swears he has no evidence now in his power of service, except the statement of Capt. (illegible name), as to his reduced circumstances ...
Sworn to before me, this day...
Some parts remain unclear due to the handwriting, but let me know if you'd like me to attempt further clarification on specific sections!"
I'm doing some genealogy work right now on my family's old papers covering the time period from recent years back to the late 17th century. Handwriting styles changed a lot over the centuries and individuals can definitely be identified by their personal cursive style of writing and you can see their handwriting change as they aged.
Then you have the problem that some of these ancestors not only had terrible penmanship but also spelled multi-syllabic words phonetically since they likely were barely educated kids who spent more time when they were young working on the farm or ranch instead of attending school where they would've learned how to spell correctly.
I don't know whether your LLM can handle English words spelled phonetically written in cursive by an individual who had no consistency in forming letters in the words. It is clear after reading a lot of correspondence from this person that they ignored things that didn't seem important in the moment like dotting i's or crossing t's or forming tails on g's, p's, j's, or even beginning letters consistently since they switched between cursive and block letters within a sentence, maybe while they paused to clarify their thoughts. I don't know but it is fascinating to take a walk through life with someone you'll never meet and to discover that many of the things that seemed awesome to you as a kid were also awesome to them and that their life had so many challenges that our generations will never need to endure.
Some of my people have the most beautiful flowing cursive handwriting that looks like the cursive that I was taught in grade school. Others have the most beautiful flowing cursive with custom flourishes and adornments that make their handwriting instantly recognizable and easy to read once you understand their style.
I think there are plenty of edge cases where LLMs will take a drunkard's walk through the scribble and spit out gibberish.
I'm reminded of an old joke though.
Ronald Reagan woke up one snowy Washington, DC morning and took a look out of the window to admire the new-fallen snow. He enjoys the beautiful scene laid out before him until he sees tracks in the snow below his window and a message obviously written in piss that said - "Reagan sucks".
He dispatched the Secret Service to the site where samples were taken of the affected snow and photos of the tracks of two people were made.
After an investigation he receives a call from the Secret Service agent in charge who tells him he has some good news and some bad news for him.
The good news is that they know who pissed the message. It was George HW Bush, his Vice President. The bad news is that it was Nancy's handwriting.
Something about extraordinary claims and extraordinary evidence? The evidence presented, a seemingly easily transcribed image, is hardly persuasive.
Some are significantly harder to read. I took the page below and tried to get GPT 4o to transcribe it and it basically couldn't do it. I'm not going to sit and prompt hack for ages to see if it can but it seems unable to tackle the handwritten text at the top. When I first just fed it the image and asked for a transcription it only (but successfully) read the bottom portion, prompted for a transcription of the top it dropped into more of a summary of the whole document mainly pulling some phrases from the bottom text. (Sadly can't share it but I copied it's reply out in a comment upthread) [0]
It was more successful at a few others I tried but it's still a task that requires manual processing like a lot of LLM output to check for accuracy and prompt modification to get it to output what you need for some documents.
https://catalog.archives.gov/id/54921817?objectPage=8&object...
[0] https://news.ycombinator.com/item?id=42746490
Ok I did one letter, from a woman in 1814 writing to James Monroe (then Secretary of State) asking for a passport to go to Scotland to get her late brother's property. What a trip! So enjoyable to get into the flow once you've "synchronized" with the persons handwriting. Furthermore, due to the fact that you're reading and re-writing word for word of whatever you're transcribing, the stories you end up reading have tremendous memory-stick. This is not surprising, considering that you are dedicating an inordinate amount of time per page, but it's a welcome side effect when you try and recollect.
> Furthermore, due to the fact that you're reading and re-writing word for word of whatever you're transcribing, the stories you end up reading have tremendous memory-stick. This is not surprising, considering that you are dedicating an inordinate amount of time per page, but it's a welcome side effect when you try and recollect.
This was something I enjoyed when I decided to learn a language by translating short stories. (Edit: Of course, you have to choose an author whose diction you respect. Your unfamiliarity with the target language encourages you to mull over the author's use of diction and the nuances the author is trying to convey, and then find appropriate diction in English. This means you spend a long time immersed in the imagery.)
What a brilliant idea. I've had learning to read French on my list for a while now, I'm going to try transcription as another way at it.
cheers! I was looking for something semi productive to sink a Friday night into
on a more serious note, working through a transcription project for letters and journals that nobody has touched since they've been archived is such a wonderful feeling. Aside from being in front of the physical document itself, your degree of separation from the writer and point is time is vanishingly small!
I always like to observe when they cross something out or make a mistake and think about what could have caused that. Did a friend pass by the door and scare them? Did they get distracted looking out the window? It's all so close and yet so far away :)
I’m interested to give this a go because I want to practice reading cursive. I do a lot of longhand writing including writing all my notes in cursive. It’s exciting to watch my binding fill up with all sorts of different subjects!
I like to write in cursive for a few reasons
1. I find it makes my hand cramp less 2. It offers some shallow privacy in public 3. I don’t want to lose the skill 4. It’s fun!
All of the same reasons I love practicing a little calligraphy! I love how it looks as well. I don’t use a special pen but just add my own style to my cursive to make it look even nicer. But I used to write my notes in school with calligraphy (mostly because it gave me an excuse to not care about the subject) but it made the teachers hate me because I would never finish copying their scribbles fast enough.
Seems like something that some of those big AI companies that are desperately starved of training material could chip in on, no? Actually do something for the public good, spend a few cents of that VC money, get some high-quality training data out of it?
This reminded me of something the historian Megan Marshall wrote in the introduction to her book The Peabody Sisters: Three Women Who Ignited American Romanticism (2005):
“I became expert in deciphering the sisters’ handwriting, and that of their ancestors, parents, and friends. Each era and each correspondent presented different challenges. Some hands were sprawling, some spindly, some cramped; t’s went uncrossed at the ends of words, and f’s and s’s were interchanged; spelling, capitalization, and punctuation could be erratic or idiosyncratic. Often, to save paper and postage, the sisters turned a single sheet ninety degrees and wrote back across a page already covered with handwriting. I learned to be especially attentive to these cross-written lines, in which the sisters invariably confided their deepest feelings in the last hurried moments of closing a letter. Here I would find the urgent personal message that had been put off for the sake of dispensing news or settling business. In one such postscript, I discovered Elizabeth’s account of a conversation with Horace Mann in which the two spoke frankly of their love for each other and finally settled on what it meant.”
A photograph of a letter with cross-writing is here:
https://www.masshist.org/database/1774
Marshall wrote more in an article for Slate:
https://slate.com/news-and-politics/2005/05/reading-the-peab...
This is all very cool so I’m not trying to be dismissive. In a lot of ways, giving a hobby out as a way to participate in the national archives is an end in itself.
But…computers can definitely do this way better, right?
I had the same thought but maybe on old hand writing they can't?
EDIT:
I tried giving the sample to 4o and it gave:
The following is the declaration of James Lambert, a soldier of the Revolutionary War in North America.
The said James Lambert this day personally appeared in the Probate Court of the County of Dearborn in the State of Indiana and at the November Term of said Court (1841), it being a court of record created by the laws of Indiana and made oath that:
On the 25th day of March 1842, he will be eighty-five years old, that he was born in the State of Maryland, that he is now a resident of said county and has been for the 27 years last past; that he has lived in Virginia, Maryland, and Pennsylvania...
> This is all very cool so I’m not trying to be dismissive. In a lot of ways, giving a hobby out as a way to participate in the national archives is an end in itself.
> But…computers can definitely do this way better, right?
No.
Cursive writing is analog and fluid, lacking consistency across authors and often inconsistent by an individual author as well. When done well, it could be classified as its own art form. When done poorly, it can resemble the path walked by a chicken on meth.
iPad seems to do OK, but it has more to go by since it has the timing and pressure as well as the written text.
Current LLMs can absolutely do this as well as you can, probably better.
> Current LLMs can absolutely do this as well as you can, probably better.
This is obviously disprovable, in that if they could, they would, and this call to action would not exist.
That's quite a lot of faith you have in them.
Them being the National Archives? What about the National Archives makes you think they're particularly inept at utilizing LLMs?
I'm tired of this brand of dismissive cynicism.
The handwriting in some of these snippets, while sometimes difficult to read for one reason or another, is nonetheless beautiful: did everyone who wrote have such great handwriting back then?
I'm looking at the piece in the Instagram post linked by the page, which begins, "honor of holding in their service". The lines are so straight, the letters are so uniform!
As someone with terrible handwriting but decent cursive, i think cursive provides a better structure for achieving cleaner penmanship compared to non-cursive writing. My theory is that cursive’s consistency of soft, flowing loops rather than a mix of abrupt angles and disconnected lines helps create a more uniform result.
I also remember teachers telling you when writing cursive to seldom lift your hand from the page. I think that act of keeping your pen on the page for most of the writing process encourages a smoother and more natural flow, reducing the chance of jerky, uneven strokes
Widespread literacy is an extremely recent phenomenon.
I highly doubt most people could write that well
Handwriting is a skill, you get better with practice!
A lot of bad handwriting stems from using it to write down things quickly (see: https://imgur.com/doctors-strike-5ANma ).
If you instead focus on doing slow calligraphy, your handwriting can improve rapidly.
After using a keyboard for circa 50 years, I can't read my own handwriting. I can't even give a reproduceable signature.
Me too, and I used to be proud of my handwriting back in the 90's. Definitely a loss in self-expression.
Isn't this like a bread-and-butter AI task?
“The following is the declaration of James Lambert, a soldier of the Revolutionary War in North America.” “The said James Lambert, on this day personally appeared in the Probate Court of the County of Dearborn in the State of Indiana, at the November Term of said Court [1841], it being a court of record created by the laws of Indiana, and made oath that on the 25th day of March 1842 he will be eighty‐five years old; that he was born in the State of Maryland; that he is now a resident of [said] county and has been for the [27] years last past; that he has lived in Virginia, Maryland, [and Pennsylvania]; that…”
These kinds of problems, matching up cursive to actual text, would seem to play to the absolute best strengths of an LLM, given how much basic language structure the models encode.
> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.
I've seen people do that, and the results are.. just sad. These modern models insert their twitter-era "what grabs attention must be true" view into the very little authentic past we still possess.
What did 4o get wrong about the title image in the transcription I just gave you?
The Australian War Memorial has a volunteer program for transcribing old letters and diaries and such: https://transcribe.awm.gov.au/
I gave it a go but it was too hard for me! I write in cursive but I found most of it illegible.
Funnily enough, there have been a few times over the past couple of years I've been asked by younger co-workers to read something for them that was written in cursive. I hadn't really realized it had become such a (comparatively) rare skill. This fact is making me feel older than my actual 50th birthday did!
I'm 28. I can only read the document in the article with a lot of effort and fiddling with the contrast.
My brother in history, I can't even read mine
I have a family heirloom civil war journal and much of it is unfortunately near undecipherable cursive writing.
It would be great if this would eventually develop into some kind of set of open models that would work on content like this.
> particularly for Americans who never learned cursive in school.
American schools don’t teach it anymore?!
Not that I can tell, unless you encounter a teacher who (personally) believes it’s worthwhile.
The real problem, IMO, is that they don’t teach cursive but also don’t teach typing. They’ve thrown laptops at the kids without giving them the basic skill necessary to be effective in that medium.
They stopped teaching cursive for a number of years but all the schools in my area start it around age 6 or 7 now. They start typing the next year with some horribly boring typing program.
Why would they? It’s an anachronism optimizing for writing speed
They started teaching it again because it correlated with better outcomes for things seemingly unrelated to writing. And it was important to learn it before typing supposedly. There is probably some better way to accomplish whatever it is actually doing, but they don't seem to know that.
It might be nice for people to be able to actually read the documents in the National Archives rather than relying on a transcription or a mobile app.
I wonder if they've considered making a simple tutorial on how to read cursive? It's not that hard if you can already read printed English. And of course you can practice on documents in the National Archives.
It's exciting and fun to learn to read an unfamiliar script, like the runes on the cover of The Hobbit ... or the engraving-style cursive of the US Constitution.
I think it likely that reading the great variety of cursive styles makes simple teaching rather complicated. Folks who spent years in school reading and writing in cursive can quickly adapt to the various styles, in a way that I'm not sure it could be done in a simple tutorial.
> I wonder if they've considered making a simple tutorial on how to read cursive?
In generations past, this was called "elementary school."
i dont think the problem is the lack of resources to learn how to read and write cursive
Except that it does say that in the article, that’s it's a lack of education in reading cursive.
Those two statements aren't at odds with each other.
For example, there's a great abundance of resources to learn about music theory and such too, the average person doesn't know such things because they aren't interested.
no, it says the opposite, that there is growing interest in bringing it back into curriculums in various states. but that's aside from the point that the smithsonian making a tutorial on reading cursive would just represent an additional resource, of which we are not lacking, to learn. whether or not we teach it is different, but finding a resource to learn is not hard.
Maybe linking to the resource. "Learn how to read this document."
I find the article's conflation of two topics involving cursive writing ignorant or disingenuous to the point that I almost wanted to respond with my own comment on that itself. If you study cursive writing in class, you are likely to learn simple and standard letterforms like Palmer script.
But the task requested by the National Archives is more akin to paleography where you can expect each author or work to have their own (region-based/family-based) handwriting that requires decipherment, even for experts. You may have encountered a coworker or schoolmate's indecipherable chicken scratch print writing; that is what you should expect, only cursive.
I don’t think I believe that OCR can’t do it but random humans can
OCR is VERY good
Actually I think in 2025 you are correct, we just haven’t got the best tech into the OCR software that’s out there in the real world. I just pasted the letter from the article into ChatGPT (4o) and asked “what does this old letter say?” The response:
—-
The following is the declaration of James Lambert, a soldier of the Revolutionary War in North America.
The said James Lambert on this day personally appeared in the Probate Court of the County of Dearborn in the State of Indiana and at the November Term of said Court (1841), it being a court of record established by the laws of Indiana and made oath that:
On the 25th day of March 1842 he will be eighty-five years old; that he was born in the State of Maryland; that he is now a resident of said county and has been for the 27 years last past; that he has lived in Virginia, Maryland, Pennsylvania…
—-
I've been trying every state of the art OCR solution on my students' handwritten essays for fifteen years and have yet to find anything even close to acceptable.
> I don’t think I believe that OCR can’t do it but random humans can
Considering the people involved are experts in their field, are certainly aware of OCR capabilities, and have publicized a need thusly:
Perhaps "random humans" can perform tasks which could reshape your belief:> OCR is VERY good
No. Sign up and look at the current missions. A lot of what they want transcribed is totally straightforward to OCR --- not even LLM, OCR. Whatever's going on, and I'm not second-guessing them, a pretty big chunk of their problem appears to be well within the state of the art. The appeal to authority isn't going to play here, because you can just click through to the archives and see what they're trying to figure out.
> No. Sign up and look at the current missions. A lot of what they want transcribed is totally straightforward to OCR --- not even LLM, OCR. Whatever's going on, and I'm not second-guessing them, a pretty big chunk of their problem appears to be well within the state of the art.
If it's that easy, then do it and be the hero they want.
Or maybe, just maybe, "a pretty big chunk of their problem appears to be well within the state of the art" is a sweeping generalization lacking understanding of the difficulties involved.
Go ahead and find something hard, and relate back the steps you took to find it.
> Go ahead and find something hard, and relate back the steps you took to find it.
This is a strawman[0] argument. You proclaimed:
And I replied: So do it or do not. Nowhere does my finding "something hard" have any relevance to your proclamation.0 - https://en.wikipedia.org/wiki/Straw_man
There are two claims. The main one is that all of these documents are easy to individually transcribe by machine. The other is that a whole lot can be OCR'd, which is pretty simple to check.
That's not a claim that processing the entire archive would be trivial. And even if it was, whether that would make someone the "hero they want" is part of what's being called into question.
So your silly demand going unmet proves nothing.
Also, "give me an example please" is not a strawman!
If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description.
> So your silly demand going unmet proves nothing.
I made demands of no one.
> Also, "give me an example please" is not a strawman!
My identification of the strawman was that it referenced "find something hard" when I had said "be the hero they want" and that what is needed in this specific problem domain may be more difficult than what a generalization addresses.
> If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description.
Maybe this is the proof you demand.
LLM's are statistical prediction algorithms. As such, they are nondeterministic and, therefore, provide no guarantees as to the correctness of their output.
The National Archives have specific artifacts requiring precise textual data extraction.
Use of nondeterministic tools known to produce provably incorrect results eliminate their applicability in this workflow due to all of their output requiring human review. This is an unnecessary step and can be eliminated by the human reading the original text themself.
Does that satisfy your demand?
> I made demands of no one.
Whatever you want to call "If it's that easy, then do it"
> LLM's [...] Does that satisfy your demand?
That's a different argument from the one above where you were trying to contradict tptacek. And that argument is flawed itself. In particular, humans don't have guarantees either.
> provably incorrect results
This gets back to the actual request from earlier, which is showing an example where the machine performs below some human standard. Just pointing out that LLMs make mistakes is not enough proof of incorrectness in this specific use case.
I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with. I'm sorry, but I declare the burden of proof here to be switched. Can you find a hard one?
(I don't think you need to Wikipedia-cite "straw man" on HN).
> I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with.
Awesome.
Can you guarantee its results are completely accurate every time, with every document, and need no human review?
> I'm sorry, but I declare the burden of proof here to be switched.
If you are referencing my stating:
Then I don't really know how to respond. Otherwise, if you are referencing my statement:> Perhaps "random humans" can perform tasks which could reshape your belief:
>> OCR is VERY good
To which I again ask, can you guarantee the correctness of OCR results will exceed what "random humans" can generally provide? What about "non-random motivated humans"?
My point is that automated approaches to tasks such as what the National Archives have outlined here almost always require human review/approval, as accuracy is paramount.
> (I don't think you need to Wikipedia-cite "straw man" on HN).
I do so for two purposes. First, if I misuse a cited term someone here will quickly correct me. Second, there is always a probability of someone new here which is unaware of the cited term(s).
> If you are referencing my stating:
> > If it's that easy, then do it and be the hero they want.
> Then I don't really know how to respond.
If someone says a thing is easy, and you respond by demanding they do it a million times to prove that it's easy, you are the one that has screwed up the burden of proof.
Can I ask, did you sign up and look at what they're actually looking for? Show of good faith: can you give 3 of the headers for the top-level "missions" they have for transcriptions?
There are conceivable reasons why they may be telling a half truth here. Just engaging the public is a worthy goal here.
> There are conceivable reasons why they may be telling a half truth here. Just engaging the public is a worthy goal here.
Asserting an ulterior motive without supporting proof is to engage in conspiracy theories.
Sometimes a cigar is just a cigar.[0]
0 - https://quoteinvestigator.com/2011/08/12/just-a-cigar/
It doesn't look like a cigar (very tricky documents) though. Hence the skepticism.
> I don’t think I believe that OCR can’t do it but random humans can
I do.
> OCR is VERY good
Uh, my experience is extremely different.
I would challenge you to find a picture of text that you think a human can read and OCR cannot. I’m happy to demonstrate. The text shown in this article is trivial.
The archivists themselves say that they run into such texts often enough that this program was needed:
> The agency uses artificial intelligence and a technology known as optical character recognition to extract text from historical documents. But these methods don’t always work, and they aren’t always accurate.
They are absolutely aware of the advances in these tools, so if they say they're not completely there yet I believe them. One likely reason is that the models probably have less 1800s-era cursive in their training set than they do modern cursive.
It's likely that with more human-tagged data they could improve on the state of the art for OCR, but it's pretty arrogant to doubt the agency in charge of this sort of thing when they say the tech isn't there yet.
Can someone please post a sample of one of these images that can only be read by a human for us naive OCR believers to see?
To be fair there was a similar discussion a few days ago in which an SME remained unconvinced: https://news.ycombinator.com/item?id=42566391
I don't necessarily agree with her conclusion because she wasn't participating directly in the thread and wasn't completely responsive to some of the points raised, but still, it appears that there are a few instances of difficult-to-read handwriting where OCR is still coming in second to skilled human interpretation.
> I would challenge you to find a picture of text that you think a human can read and OCR cannot.
Are you aware of CAPTCHA[0] images?
0 - https://en.wikipedia.org/wiki/CAPTCHA
Text that is _intentionally constructed_ to fool computers but not humans is obviously out of scope. But they’re generally easily solved with OCR these days anyway.
Solvable with the right tools.
https://github.com/noCaptchaAi/NoCaptcha-Ai-Browser-Extensio...
> Solvable with the right tools.
The original assertion was:
Not if many CAPTCHA image challenges could be automated. Unless the tool referenced guarantees 100% correct solutions for all manipulated text images.Your experience is obsolete.
Oh, ok then.
I mean, all you have to do is feed the image to ChatGPT, and it will read it basically as well as you can.
Denying/downvoting reality is always an option, of course.
Not being rude was also an option, one you chose not to take for some reason. Seriously, all it would've taken was for you to say something like "there have been a lot of advancements so it's probably different than you remember". This conversation would've gone much smoother for you if you had.
And BugsJustFindMe can't downvote you, because it was a reply to him. So don't bite his head off over it. You got downvoted because you were a jerk, plain and simple.
Not being rude was also an option
Refraining from reflexively pooh-poohing AI with uninformed and/or out-of-date opinions is also an option, but not one often exercised on HN.
It gets old not being able to carry on a discussion without squinting at grayed-out text, simply because someone pointed out that humans aren't robots and should no longer have to emulate them.
Why do they need volonteers to manually do it? Open AI models like Microsoft's TrOCR are very effective for handwritten English
It says "The following is the dedication of James Lambert a soldier of the Revolutionary wars with the Americas."
blah blah blah
I'm not too sure about that reading, I got "The following is the declaration of James Lambert a soldier of the Revolutionary War in South America." rather different
oh, it is "declaration", yes, but not South America. this guy is even on Amazon:
https://www.amazon.com/James-Lambert-1758-1847-Elaboration-R...
I got “North America”
Hmm, interesting: “North America” does make sense, and 4o also seems to transcribe it that way, but the handwriting looks like it says “South America” to me.
Yes, that seems right. Not that difficult. This one suffers from some poor penmanship, though.