You could potentially build on stable Rust by emitting the breakpoint instructions yourself, at least on popular platforms. For instance, `core::arch::asm!("int3")` on x86, or `core::arch::asm!("brk #1")` on ARM.
Also, this is providing motivation to want to stabilize a breakpoint mechanism, perhaps `core::arch::breakpoint()`. I'm going to propose an API Change Proposal (ACP) to the libs-api team to see if we can provide that in stable Rust.
Plain int3 is a footgun: the CPU does not keep track of the address of the int3 (at least not until FRED), and it reports the address after int3. It’s impossible to reliably undo that in software, and most debuggers don’t even try, and the result is a failure to identify the location of the breakpoint. It’s problematic if the int3 is the last instruction in a basic block, and even worse if the optimizer thinks that whatever is after the int3 is unreachable.
If Rust’s standard library does this, please consider using int3;nop instead.
Good to know! I've seen the pattern of "int3; nop" before, but I've never seen the explanation for why. I'd always assumed it involved the desire to be able to live-patch a different instruction over it.
In Rust, we're using the `llvm.debugtrap` intrinsic. Does that DTRT?
The "canonical" INT 3 is a single byte opcode (CCh), so the debugger can just subtract 1 from the address pushed on the stack to get the breakpoint location.
There is another encoding (CD 03), but no assembler should emit it. It used to be possible for adversarial code to confuse debug interrupt handlers with this, but this should be fixed now.
This would involve the debugger actually being structured in a way that makes this make sense. A debugger like GCC has a gnarly data structure that represents the machine state, and it contains things like EIP/RIP. There is a command 'backtrace' that takes the machine state and attempts to generate a backtrace. And there's a command 'continue' that resumes execution.
int3 is a "trap". continue will resume execution at the instruction after int3, as intended. But backtrace should, by some ill-defined magic, generate the backtrace as though RIP was (saved RIP - 1). And the condition for doing this isn't something that is (AFAIK) representable at all in GCC's worldview. Sure, GCC knows, or at least ought to know [0], that it gained control because of vector 3, and the Intel and AMD manuals say that vector 3 is a trap. But there isn't a bit in memory or anything you would see in 'info regs' that will say "hey, this is a 'trap', and backtraces and such should be done as though RIP was actually RIP-1".
Maybe the right solution would be to split the program counter, from the perspective of the debugger, into two fields: program counter for backtracing, and program counter for resumption.
And yes, I know that GCC gets this wrong. Been there, seen the failures. I have not checked, but I expect that LLDB works exactly like GCC in this regard.
[0] ptrace on Linux exposes the vector number, somewhat awkwardly. Or you can infer it from the fact that the signal was SIGTRAP.
Having a feature like this will significantly increase the demand of better incremental compilation, potentially with the need for patching specific items on existing binaries for speed. At that point you could get very close to an IDE debugging experience/speed with only rustc, a text editor and a debugger. (Inserting a bunch of NOPs on every function and supporting patching of JMPs in their place would likely go a long way for this.)
Thanks, yeah I considered using the instructions directly, but I was hoping for a more cross-platform option. For my purposes, developing in the Bevy engine, nightly isn't a huge blocker. Yeah, it would be really great to just have breakpoint support in stable Rust, thanks for doing the proposal! I'll consider stable support in the meantime.
On Unix platforms, you could just raise SIGTRAP directly, it will pause the attached debugger and works regardless of architecture.
This is the macro I use for example:
#[doc(hidden)]
pub use libc as __libc;
// This is a macro instead of a function to ensure the debugger shows the breakpoint as being at
// the caller instead of this file.
#[cfg(unix)]
#[macro_export]
macro_rules! breakpoint {
() => {
unsafe {
use $crate::__libc as libc;
libc::raise(libc::SIGTRAP);
}
};
}
Interesting. Unfortunately, I'm not well versed in assembly, is there a similar trick or requirement in arm and would that include Apple silicon, or is this something specific to `int3` on x86? That may explain why it was inconsistent during my development process, I didn't think to check if the inconsistency was platform dependent.
Answering my own question, apparently `brk #1` is insufficient on Apple silicon. That results in just a trap and will prevent the debugger from continuing past the debug statement. From a bit of searching and my experiments `brk #0xF000` was the way to go instead which had the consequence of not always landing on the debug statement, the addition of a nop with `brk #0xF000 \n nop` resulted in the debugger landing on the correct statement.
Rusts current pretty printers in lldb and gdb are just not good enough for a fluid step debugging experience. I've had luck with intellij IDE's but it's very sad that the best we can do in a language with such good devex tooling is print debugging.
Theoretically you could generate debug scripts with a macro and embed them with
I find myself using debug_here (https://github.com/ethanpailes/debug-here) which does similar things but for desktop-class debugging. It hasn't been updated in a while, but it still works for me on Windows at least.
Neat project! Maybe this decision is copied over from unreal engine, but instead of `ensure` and `ensure_always`, having names like `ensure_once` and `ensure` would have been more clear to me.
I can understand where you're coming from, but when programming games you generally don't want a breakpoint to be hit more than once since you are running a loop over multiple frames. So in this case the concept of ensure_once is more common, so the shorter inverse is more convenient. Asserts should be enough to get your attention and not to annoy, so orienting it this way is a deliberate choice.
The master branch of the Rust repo is built every night and distributed. That's a nightly build. Most people are on the stable release, which is updated every six weeks.
A minority use the nightly build for various reasons: a feature that hasn't reached stable yet, or because they want to help test the nightly releases and prevent bugs from reaching stable.
89.4% of users surveyed use stable. 31.1% use nightly, but there's overlap there (e.g. I use nightly to try out new things but don't build things that depend on nightly). Only 11.5% of people say they use a crate that requires it.
> Only recently have I had some projects switching to stable after their required features stabilized.
While this used to be very common back then (in 2016-18 many things where progressively stabilized and you had cornerstone libraries switching to stable at almost every release) it hasn't been so for the past 5 years.
Rust gets way less “exciting” new features nowadays, as the language has settled a lot (there are still moving parts, but they are the minority not the norm).
I assume any "nightly" version would work in this context, meaning it would not refer to a version from a year ago, as it would have already been made stable by that point, right?
"nightly" versions also allow to use unstable features, and unstable features may remain so for a very long time (potentially forever) without breaking, so an old nightly could maybe work
Just use a recent nightly and you should be fine. Rust project doesn't offically provide support even for old stable versions, so "using nightly" with no specifics generally means using any nightly build, around or newer than, the current stable release.
A lot of people pin a specific nightly since the feature they're depending on could change (or ostensibly it would be in stable), so they can keep working without continuously having to track nightly changes or deal with it breaking on a different system with a different nightly version.
That obviously only works for non-library uses of nightly.
It's a bit unfortunate wording but it basically requires any nightly toolchain version. It uses `std::intrinsics::breakpoint()` which is a compiler intrinstic. This has been available for a long time, but afaik will never be exposed on a stable toolchain.
I didn't come across any good ones when creating this library, but if you're using VSCode, I tried my best to make the README as beginner friendly as possible. I'm open to issues and PRs if anything is unclear. I think part of the issue is that debugging is not yet very common in the Rust ecosystem, partially due to the excellent borrow checker and error messages, but partially due to immature tooling, hence I made this to promote the practice of debugging.
You could potentially build on stable Rust by emitting the breakpoint instructions yourself, at least on popular platforms. For instance, `core::arch::asm!("int3")` on x86, or `core::arch::asm!("brk #1")` on ARM.
Also, this is providing motivation to want to stabilize a breakpoint mechanism, perhaps `core::arch::breakpoint()`. I'm going to propose an API Change Proposal (ACP) to the libs-api team to see if we can provide that in stable Rust.
Plain int3 is a footgun: the CPU does not keep track of the address of the int3 (at least not until FRED), and it reports the address after int3. It’s impossible to reliably undo that in software, and most debuggers don’t even try, and the result is a failure to identify the location of the breakpoint. It’s problematic if the int3 is the last instruction in a basic block, and even worse if the optimizer thinks that whatever is after the int3 is unreachable.
If Rust’s standard library does this, please consider using int3;nop instead.
Good to know! I've seen the pattern of "int3; nop" before, but I've never seen the explanation for why. I'd always assumed it involved the desire to be able to live-patch a different instruction over it.
In Rust, we're using the `llvm.debugtrap` intrinsic. Does that DTRT?
The "canonical" INT 3 is a single byte opcode (CCh), so the debugger can just subtract 1 from the address pushed on the stack to get the breakpoint location.
There is another encoding (CD 03), but no assembler should emit it. It used to be possible for adversarial code to confuse debug interrupt handlers with this, but this should be fixed now.
This would involve the debugger actually being structured in a way that makes this make sense. A debugger like GCC has a gnarly data structure that represents the machine state, and it contains things like EIP/RIP. There is a command 'backtrace' that takes the machine state and attempts to generate a backtrace. And there's a command 'continue' that resumes execution.
int3 is a "trap". continue will resume execution at the instruction after int3, as intended. But backtrace should, by some ill-defined magic, generate the backtrace as though RIP was (saved RIP - 1). And the condition for doing this isn't something that is (AFAIK) representable at all in GCC's worldview. Sure, GCC knows, or at least ought to know [0], that it gained control because of vector 3, and the Intel and AMD manuals say that vector 3 is a trap. But there isn't a bit in memory or anything you would see in 'info regs' that will say "hey, this is a 'trap', and backtraces and such should be done as though RIP was actually RIP-1".
Maybe the right solution would be to split the program counter, from the perspective of the debugger, into two fields: program counter for backtracing, and program counter for resumption.
And yes, I know that GCC gets this wrong. Been there, seen the failures. I have not checked, but I expect that LLDB works exactly like GCC in this regard.
[0] ptrace on Linux exposes the vector number, somewhat awkwardly. Or you can infer it from the fact that the signal was SIGTRAP.
I assume you meant GDB, not GCC, right?
Seems like a deficiency in GDB (and maybe LLDB too), not in the kernel or x86.
I do mean GCC. Whoops.
Deficiency or not, it breaks debugging. I’m willing to pay a cost of one byte per breakpoint as a workaround.
And GDB has far more outrageous, if less-frequently hit, bugs in its architectural state handling. I’m not holding my breath for a fix.
Having a feature like this will significantly increase the demand of better incremental compilation, potentially with the need for patching specific items on existing binaries for speed. At that point you could get very close to an IDE debugging experience/speed with only rustc, a text editor and a debugger. (Inserting a bunch of NOPs on every function and supporting patching of JMPs in their place would likely go a long way for this.)
Thanks, yeah I considered using the instructions directly, but I was hoping for a more cross-platform option. For my purposes, developing in the Bevy engine, nightly isn't a huge blocker. Yeah, it would be really great to just have breakpoint support in stable Rust, thanks for doing the proposal! I'll consider stable support in the meantime.
On Unix platforms, you could just raise SIGTRAP directly, it will pause the attached debugger and works regardless of architecture.
This is the macro I use for example:
Hah, the README says:
> Additonally, debugging may not land on the macro statements themselves.
See my comment above, and give int3;nop a try.
Interesting. Unfortunately, I'm not well versed in assembly, is there a similar trick or requirement in arm and would that include Apple silicon, or is this something specific to `int3` on x86? That may explain why it was inconsistent during my development process, I didn't think to check if the inconsistency was platform dependent.
Answering my own question, apparently `brk #1` is insufficient on Apple silicon. That results in just a trap and will prevent the debugger from continuing past the debug statement. From a bit of searching and my experiments `brk #0xF000` was the way to go instead which had the consequence of not always landing on the debug statement, the addition of a nop with `brk #0xF000 \n nop` resulted in the debugger landing on the correct statement.
Rusts current pretty printers in lldb and gdb are just not good enough for a fluid step debugging experience. I've had luck with intellij IDE's but it's very sad that the best we can do in a language with such good devex tooling is print debugging.
Theoretically you could generate debug scripts with a macro and embed them with
but I haven't seen anyone go through the trouble.The pretty printers seem to break from time to time as well with compiler updates: https://www.reddit.com/r/rust/comments/1f28akn/invalid_value...
I find myself using debug_here (https://github.com/ethanpailes/debug-here) which does similar things but for desktop-class debugging. It hasn't been updated in a while, but it still works for me on Windows at least.
Oh, that’s super clever integration with tracing. Feel free to send us a PR; we’ll link to this as a related project in the docs!
PR opened: https://github.com/tokio-rs/tracing/pull/3147
Thanks! I'll get a PR up shortly!
Neat project! Maybe this decision is copied over from unreal engine, but instead of `ensure` and `ensure_always`, having names like `ensure_once` and `ensure` would have been more clear to me.
I can understand where you're coming from, but when programming games you generally don't want a breakpoint to be hit more than once since you are running a loop over multiple frames. So in this case the concept of ensure_once is more common, so the shorter inverse is more convenient. Asserts should be enough to get your attention and not to annoy, so orienting it this way is a deliberate choice.
Ah makes sense, thank you
What does nightly mean ? I hate that you could not know a specific version of a nightly.
The master branch of the Rust repo is built every night and distributed. That's a nightly build. Most people are on the stable release, which is updated every six weeks.
A minority use the nightly build for various reasons: a feature that hasn't reached stable yet, or because they want to help test the nightly releases and prevent bugs from reaching stable.
Is it a minority? Are there stats posted for this?
Only recently have I had some projects switching to stable after their required features stabilized.
https://raw.githubusercontent.com/rust-lang/surveys/main/sur...
89.4% of users surveyed use stable. 31.1% use nightly, but there's overlap there (e.g. I use nightly to try out new things but don't build things that depend on nightly). Only 11.5% of people say they use a crate that requires it.
> Only recently have I had some projects switching to stable after their required features stabilized.
While this used to be very common back then (in 2016-18 many things where progressively stabilized and you had cornerstone libraries switching to stable at almost every release) it hasn't been so for the past 5 years.
Rust gets way less “exciting” new features nowadays, as the language has settled a lot (there are still moving parts, but they are the minority not the norm).
You can pin versions with the rust-toolchain.toml file you need to be using Rustup afaik. Nightly is just the daily builds.
I thought nightly is for nightly builds. I'll get my coat.
I assume any "nightly" version would work in this context, meaning it would not refer to a version from a year ago, as it would have already been made stable by that point, right?
"nightly" versions also allow to use unstable features, and unstable features may remain so for a very long time (potentially forever) without breaking, so an old nightly could maybe work
Right, not everything gets merged to stable. In that case: letting us know the specifics beyond "nightly" is advisable, IMO.
The readme does mention the specifics, immediately after mentioning nightly.
> BREAKPOINTS REQUIRE ENABLING THE EXPERIMENTAL core_intrinsics FEATURE
How do I know which versions of nightly support that feature, and that specific version of the feature though?
I like your username.
Just use a recent nightly and you should be fine. Rust project doesn't offically provide support even for old stable versions, so "using nightly" with no specifics generally means using any nightly build, around or newer than, the current stable release.
A lot of people pin a specific nightly since the feature they're depending on could change (or ostensibly it would be in stable), so they can keep working without continuously having to track nightly changes or deal with it breaking on a different system with a different nightly version.
That obviously only works for non-library uses of nightly.
It's a bit unfortunate wording but it basically requires any nightly toolchain version. It uses `std::intrinsics::breakpoint()` which is a compiler intrinstic. This has been available for a long time, but afaik will never be exposed on a stable toolchain.
Per https://dev-doc.rust-lang.org/nightly/unstable-book/library-...
>This feature is internal to the Rust compiler and is not intended for general use.
We used to divide by zero but then someone decided that was ok.
Is there a good newby tutorial on how to use debugger with Rust (and debugger in general?) No videos please.
I didn't come across any good ones when creating this library, but if you're using VSCode, I tried my best to make the README as beginner friendly as possible. I'm open to issues and PRs if anything is unclear. I think part of the issue is that debugging is not yet very common in the Rust ecosystem, partially due to the excellent borrow checker and error messages, but partially due to immature tooling, hence I made this to promote the practice of debugging.