There are `asprintf` and `vasprintf` (takes a va_list argument). Those allocate a sufficiently sized buffer that can be released with `free`.
Yes, it's a GNU extension, but it's also supported by various BSDs [1][2][3], and yes, Musl has it too. It's present in pretty much any sane C library.
And combine it with __attribute__((cleanup)) to get the string automatically freed at the end of your function (if that's the right thing to do). Looks like cleanup with be standardized finally in the next C2x.
> And combine it with __attribute__((cleanup)) to get the string automatically freed at the end of your function (if that's the right thing to do). Looks like cleanup with be standardized finally in the next C2x.
The problem is that on error the buffer pointer value is undefined, so you can't just unconditionally call free on the pointer. There's at least one proposal for C2x that avoids adopting asprintf for this reason despite it already being added to POSIX.
This undefined'ness is a vestige of the original glibc implementation. The proper solution is to either require that the pointer value be preserved on error (thus preserving NULL if the caller initialized it) or require the implementation set it to NULL. IIRC, when added by the BSDs (1990s) and later Solaris they explicitly documented it to set the pointer to NULL. And it seems that late last year glibc finally adopted this behavior as well.[1]
Another tip: don't use normal asprintf as-is, but write your own very similar helper!
1. have it free the passed-in buffer, so that you can reuse the same pointer
2. have it do step 1 after the formatting, so the old value can be a format argument
3. when getting the size of the full expansion, don't format to NULL, but do it to a temp buffer (a few KB in size) - then if the expansion is small enough, you can skip the second format into the actual buffer. Just malloc and memcpy. You know how many chars to memcpy, because that's the return value from snprintf
> Thanks, first I've heard of them and they happen to solve a real problem I'm working on today. Always nice when you can learn something new...
You don't really need to, TBH. I pretty much always wrote a malloc-memory `sprintf` alternative if the system didn't have one. it's only a few lines of code, that'll take maybe 10m of your day the first time you realise `sprintf` on the platform doesn't exist.
I know, that is what I was planning on doing. (and might be what I end up doing anyway since I need to truncate the utf-8 string if it is > 1024 bytes...) Still it is nice to have other options - this code is run in some tight loops so I will be profiling all the options.
I've actually learned a few little tricks reading the fucking gcc manual. If you're coding C (or C++) regularly, the manual is a good learning source and is well-written.
That little-known feature turns out to be crucial if you're not careful. printf returns how many bytes were printed. snprintf returns how many bytes would have been printed had the buffer been large enough. Useful for sizing your buffer, but making it a dangerous printf replacement if you don't know the difference.
Cisco and many of Ciscro's customers found out the hard way (during CitrixBleed, https://www.assetnote.io/resources/research/citrix-bleed-lea...), leaking random blocks of memory in the proprietary, C-based web server of their security appliance that gets compromised every now and then.
Always read the docs of your system. All of the xxprintf functions are not the same. They are sneaky and look the same on the surface. Even silly things like what the % items do can vary between platforms, or have different meanings, or be missing all together.
This has annoyed me a lot, crazy you can't rely on the print functions being the same on different platforms. This is why I always add stb_sprintf and make an allocSprintf() that allocates the right sized buffer first.
> If the output was truncated due to this limit, then the return value is the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available.
The initial call with size 0 tells you the necessary length of the buffer for the string you want, but does not include the null byte.
For clarity, all snprintf calls "return the number of bytes that would be written to s had n been sufficiently large excluding the terminating null byte" [1].
I’d argue this is one of the cursed design choices of the standard library.
Way to easy to use the returned value as the “actual length” of the written string. Sure, that was never the intent, but still…
The standard library also makes appending a formatted string to an existing one surprisingly nontrivial…
What should be a 1-liner is about 5-10 lines of code (to include error handling) and is somewhat hard to read. The “cognitive load” for basic operations shouldn’t be high…
That reminds me of an article where the author was like the most disastrous design choices in all of programming, was the NULL-terminated string. It's telling that no other language since C really does that.
There are `asprintf` and `vasprintf` (takes a va_list argument). Those allocate a sufficiently sized buffer that can be released with `free`.
Yes, it's a GNU extension, but it's also supported by various BSDs [1][2][3], and yes, Musl has it too. It's present in pretty much any sane C library.
[1] https://man.openbsd.org/man3/printf.3
[2] https://man.netbsd.org/vasprintf.3
[3] https://man.freebsd.org/cgi/man.cgi?query=vasprintf&sektion=...
And combine it with __attribute__((cleanup)) to get the string automatically freed at the end of your function (if that's the right thing to do). Looks like cleanup with be standardized finally in the next C2x.
> And combine it with __attribute__((cleanup)) to get the string automatically freed at the end of your function (if that's the right thing to do). Looks like cleanup with be standardized finally in the next C2x.
The problem is that on error the buffer pointer value is undefined, so you can't just unconditionally call free on the pointer. There's at least one proposal for C2x that avoids adopting asprintf for this reason despite it already being added to POSIX.
This undefined'ness is a vestige of the original glibc implementation. The proper solution is to either require that the pointer value be preserved on error (thus preserving NULL if the caller initialized it) or require the implementation set it to NULL. IIRC, when added by the BSDs (1990s) and later Solaris they explicitly documented it to set the pointer to NULL. And it seems that late last year glibc finally adopted this behavior as well.[1]
[1] https://sourceware.org/git/?p=glibc.git;a=commit;h=cb4692ce1...
Another tip: don't use normal asprintf as-is, but write your own very similar helper!
1. have it free the passed-in buffer, so that you can reuse the same pointer
2. have it do step 1 after the formatting, so the old value can be a format argument
3. when getting the size of the full expansion, don't format to NULL, but do it to a temp buffer (a few KB in size) - then if the expansion is small enough, you can skip the second format into the actual buffer. Just malloc and memcpy. You know how many chars to memcpy, because that's the return value from snprintf
(Don't forget to check for errors and all that.)
asprintf and vasprintf are part of POSIX, now.
Thanks, first I've heard of them and they happen to solve a real problem I'm working on today. Always nice when you can learn something new...
> Thanks, first I've heard of them and they happen to solve a real problem I'm working on today. Always nice when you can learn something new...
You don't really need to, TBH. I pretty much always wrote a malloc-memory `sprintf` alternative if the system didn't have one. it's only a few lines of code, that'll take maybe 10m of your day the first time you realise `sprintf` on the platform doesn't exist.
Here is a sample from more recently: https://github.com/lelanthran/libds/blob/b5289f6437b30139d42...
I know, that is what I was planning on doing. (and might be what I end up doing anyway since I need to truncate the utf-8 string if it is > 1024 bytes...) Still it is nice to have other options - this code is run in some tight loops so I will be profiling all the options.
Came here to say exactly this.
The lost art of RTFM.
I've actually learned a few little tricks reading the fucking gcc manual. If you're coding C (or C++) regularly, the manual is a good learning source and is well-written.
[dead]
That little-known feature turns out to be crucial if you're not careful. printf returns how many bytes were printed. snprintf returns how many bytes would have been printed had the buffer been large enough. Useful for sizing your buffer, but making it a dangerous printf replacement if you don't know the difference.
Cisco and many of Ciscro's customers found out the hard way (during CitrixBleed, https://www.assetnote.io/resources/research/citrix-bleed-lea...), leaking random blocks of memory in the proprietary, C-based web server of their security appliance that gets compromised every now and then.
Always read the docs of your system. All of the xxprintf functions are not the same. They are sneaky and look the same on the surface. Even silly things like what the % items do can vary between platforms, or have different meanings, or be missing all together.
This has annoyed me a lot, crazy you can't rely on the print functions being the same on different platforms. This is why I always add stb_sprintf and make an allocSprintf() that allocates the right sized buffer first.
New nightmare scenario unlocked.
This sentence is confusing:
> I have size_with_nul because snprintf man pages say
> The functions snprintf() and vsnprintf() write at most size bytes (including the terminating null byte (‘\0’)) to str.
If 'size' includes the null byte, why do we have to add 1?
> If the output was truncated due to this limit, then the return value is the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available.
The initial call with size 0 tells you the necessary length of the buffer for the string you want, but does not include the null byte.
For clarity, all snprintf calls "return the number of bytes that would be written to s had n been sufficiently large excluding the terminating null byte" [1].
1. https://man7.org/linux/man-pages/man3/fprintf.3p.html
I’d argue this is one of the cursed design choices of the standard library.
Way to easy to use the returned value as the “actual length” of the written string. Sure, that was never the intent, but still…
The standard library also makes appending a formatted string to an existing one surprisingly nontrivial…
What should be a 1-liner is about 5-10 lines of code (to include error handling) and is somewhat hard to read. The “cognitive load” for basic operations shouldn’t be high…
That reminds me of an article where the author was like the most disastrous design choices in all of programming, was the NULL-terminated string. It's telling that no other language since C really does that.
I think this was it: https://queue.acm.org/detail.cfm?id=2010365
True.
The number can be determined at comp-time.
Buffer sizes should be computing manually.
The first sentence is confusing for me.
Once I hacked up an snprintf wrapper that automated any required realloc calls:
https://github.com/RhysU/snprintf_realloc/blob/master/snprin...
Worth critically reviewing before using. It's been a while.