The main issue I see is that papers are actually becoming so focussed on form that they are now unreadable. People prefer reading my blog for my papers than reading the papers themselves. In fact I hear people telling me they understood the blog _better_. The whole academic writing shtick has become so obtuse that not only writing is cumbersome, but so is reading.
The other side of all this academic brownie points via papers (and doing reviews, which has become "brownie points for gatekeeping") is that most academic software is not only unmaintained, but actually unusable. They rarely even compile, and if they do, there is no --help, no good defaults, no README, and no way to maintain them. They are single-use software and their singular use is to write the paper. Any other use-case is almost frowned upon.
One of the worst parts of Academic software is that if you re-write it in a ways that's actually usable and extensible, you can't publish that -- it's not new ("research") work. And you will not only have to cite the person who wrote the first useless version forever, but they will claim they have done it if your tool actually takes off.
BTW, there are academics who don't follow this trend. I am glad that in my field (SAT), some of the best, e.g. Armin Biere and Randal Bryant are not like this at all. Their software is insanely nice and they fix bugs many-many years after release. Notice that they are also incredibly good engineers.
> In fact I hear people telling me they understood the blog _better_. The whole academic writing shtick has become so obtuse that not only writing is cumbersome, but so is reading.
Legible writing has little incentive. You can write in a simple down-to-earth manner and risk having someone objecting "Your work is isomorphic to X, done years ago." Your counter-objection, "Well, X is so incomprehensible that no one had any chance of understanding it", will fall flat. Better not risk it, write in an obtuse form and withhold the source code.
there is only 1 implementation available for an algorithm on which i did part of my bs internship, and it uses a lot of harcoding. It of course makes benchmarking and extending it much, much harder if you want to change some things
Unfortunately academic resource doesn't seem to be focused on the hard problems of today. A fast algorithm is nice, but we need to be able to maintain code long term, understand code, and fix bugs. I have 15 million lines of code I'm supposed to maintain and I know many others reading this work on larger projects. There is no way one human can write that much software. There is no way we can afford to throw it away even if there is a better way (well we could, but based on the last time we did that the cost would be over 1 billion dollars and 8 years - not a useful investment since there is no reason to think anything will make those prices go down). We need to keep it working.
This is a shock to many of our leaders - who were writing 8 bit assembly to do similar things. They commonly did throw away all the work of the last version since it only took them a few months to rewrite it for the exact features they needed. (having experience because they wrote it just a year ago means the rewrite as much faster, and the limitations of 8 bit means it was worth rewriting since they had to remove one feature to add a new one).
Not to mention, academia likes to think it is isolated from the real world, which is advertising and marketing all the way down. We should probably mint more dual CS and marketing majors.
That's not a part of computer science that's organizational and process management at that point not computer science so I'm not shocked CS academics aren't interested in it. You're looking at the wrong group to answer your question, we have a great amount of discussion about how to organize large software to make it manageable already too but most of it comes down to breaking it into more manageable chunks with defined interfaces and testing that interface thoroughly.
Fir enough if the other group was looking at it but they are not from what I can tell.
ane frankly CS without those skills is useless for the reason I was replying to. If they cannot make readable/maintainable code they need to get a different job.
There's loads of space in a CS career where you'll never get anywhere near responsibility for a hundred thousand much less multiple millions of lines of code at once.
And there are already loads of advice out there about how to organize and manage huge code bases. It's usually consultancy based for anything but the broadest ideas though because it's so specific to the particular situation. An architecture is as much defined by the application it's built for as it is by what people would like to impose on it for convenience.
The article says that Attention is All You Need wasn't published in a "standard" conference, but wasn't it published in NIPS? Did I misread the article?
A big part of the problem here is that Universities have increasingly begun attaching prestige to specific “top” conference publications for both ranking and faculty promotions. A good example of the phenomenon can be seen in [1] (sorry for the noun-citation!) which only gives credit for approximately three conferences in each field. Combine this with a flood of new researchers entering CS, you have a recipe for “top” conferences being essentially destroyed and filled with uninspired work.
(And contrary to the joke in the article, even your own work becomes uninspired when you ship it to those conferences. You can’t afford to be quirky or interesting.)
Fortunately every field has a fourth or fifth-tier conference that isn’t on this list (or a specialized topic conference that the rankings folks don’t care about), and those still serve the purposes that conferences were made for. You just might not be able to convince a ranking-obsessed administrator that your work has any value if you publish there.
(I imagine you agree, so this is just to expand) a secondary, insidious issue is that administrators diffuse their rules through the bureaucracy. In the case of CS, you start seeing references to csrankings in recommendation letters for grad applications, faculty applications, or even tenure letters. At that point, it can be hard to fight against it.
Note how the author talks about "main conferences", "major conferences", and "top conferences". That's the root issue. Whenever there is prestige available, people will compete for it. And if you have a competition, you should formalize the rules to make it fair.
When I was doing PhD ~15 years ago, I noticed that I rarely cited work that appeared in the top conferences of the subfield. Those conferences covered so wide range of topics that often only 1 or 2 papers were in the same subsubfield as me. And even those were often not directly relevant to my work.
But then there were small specialized conferences that had plenty of interesting papers every year. I left CS for another field a decade ago, but I still regularly attend some of those conferences and review for them. The papers published in them are still interesting and relevant to my work.
This article resonated with me and puts into words some of the feelings I had towards the end of my PhD. This part:
> An interesting case in software engineering is dismissal for lack of “evaluation.” It would be, of course, ridiculous to deny the benefits that the emphasis on systematic empirical measurement has brought to software engineering in the last three decades. But it has become difficult today to publish conceptual work not yet backed by systematic quantitative studies.
struck a chord with me. The top-tier CS systems conferences for me (OSDI and SOSP) have gotten to the point where you basically have to be writing the paper about the system you built at a FAANG that serves 1B users daily to get accepted.
It's hard for a novel idea and first-cut implementation to compete with systems built over many years with a team of a dozen software engineers. Obviously, those big systems deserve tons of credit and it's amazing that Big Tech publishes those papers! Credit to them. But it's also the case that novel ideas with an implementation that hasn't seen 1B users yet still have value.
I suppose the argument is that workshops serve that purpose of novel ideas with unproven implementation. There's some truth to that, but as the article highlights, the full conference papers are the real currency.
I think the example of how to "correctly" cite a paper actually makes this issue seem smaller than it is. In reality, these conferences have very complicated (and unstated) "rules" for how a paper is supposed to look. If an "outsider" wanders in and submits a paper with new ideas, it will be very obvious that they are not a "member of the community" and their paper will usually be treated much more harshly as a result. This adds a huge amount of friction to research.
And what's particularly frustrating is that many organizers will try to combat this by writing papers saying they "particularly encourage" papers that are interdisciplinary, or focused on less fashionable topics, etc. It's good that they are trying to change things, but I think the main effect in practice is to encourage people to spend their time writing papers that have little chance of being accepted.
This issue isn't at all unique to computer science, though. Try publishing a paper in a top economics journal as an outsider!
I am fairly certain this rule was there against an obnoxious citing style of "The lambda calculus [1] was intended as a foundation for mathematics". It is especially obnoxious in the case of CS because when you cite e.g. "as Johns comments in his article about future developments of the programming languages [1963a]" it is quite important to know that this paper is actually from 1963 and can be mostly disregarded except as a historic curiosity; yet I've seen people vehemently defending this "[1]" style.
Is citation style really an issue? Even if they don't state which style they expect, surely you can tell their expected style from their existing publications? With proper tooling (e.g. LaTeX+BibTeX) it's pretty painless to switch styles.
Here's that rant of a blog from D.J. Bernstein [0] about how "[3, 7, 42]" citation style is superior and promotes scientific progress that I was thinking about when I wrote my comment. I personally find most of his reasoning pretty unconvincing; and so while I understand Meyer's irritation, I have to say I have to side with OOPSLA here. After all, you'd also probably want the submitted papers to be written in somewhat better than 5th-grade-high-school-student's English, and don't have way too many typos (I talk like ~15 typos per page).
Opinions vary on citation styles, my point was that it seems reasonable for a publication to standardise on one citation style, i.e. to require its use. I'm not sure it's what dynm meant by very complicated (and unstated) "rules" for how a paper is supposed to look. That article by DJB mentions that every author really ought to be using a citation-management solution like BibTeX, so that regardless of your preferences, it's easy to change your whole paper to a different citation style.
There's a slight (but only slight) irony in your use of the HackerNews convention for handling multiple links without breaking up the body of the main text. In this short-form medium it works great. I see someone made this same point in the thread you linked, at https://news.ycombinator.com/item?id=40673426
The only time I like numbers is writing proposals and I only like it because it saves space. Other than that I much prefer (name, year) if I am to have a preference at all.
Adding to the frustration, (the lack of) these shibboleths partially undermine double-blind reviewing, which is on the rise in prestigious conferences. A reviewer from the in-group may immediately spot that a submission comes from the out-group.
When I started my PhD in CS/AI in the late 90s, my department's AI faculty was already telling us, "at the major conferences, all the action is in the workshops." And this was my experience, the workshops were indeed where you found the most engaging experiences and interesting new things.
Meanwhile the work at the main at the main conference of AAAI or ICML was much farther along, and the value of having it presented at a conference, rather than a journal, was minimal. The conventional wisdom was, "the talk is just an advertisement for the paper."
There’s a larger problem here beyond careerization that is going unsaid. People are using these metrics as an assessment of their self worth. I’ve had top researchers point out that such and such publication was accepted in the industrial track vs research track, and how important it was to keep that flag in the bibliographical data, even well after publication in a top conference. Most papers research or industrial stay in obscurity. The truly novel ones will catch like wildfire. The metricization of research and academic standing is the underlying culprit.
This doesn't always happen; there are many historical examples where the first publication of an idea or technology goes unnoticed, often for many years, until someone better-positioned re-invents the same thing.
Yes papers not having working code is very annoying and anti intellectual. I wrote a substack about this recently because I review a lot of papers that essentially are trash because the authors do not provide working codes:
1) Increase page length maximums, 10 pages is too low to explain something. I am looking directly at you IEEE.
2) Record and post all of the conferences, ACM is doing great strides here, but many many conferences aren't even recorded.
3) Reduce the paywalls and the access to all information, again I want to really support the ACM here, but the conferences that aren't broadcast for free (and also aren't recorded) are also the ones that are hella expensive and you have to attend in person (making them 5x as expensive with travel and lodging). Provide a happy medium, $300 for remote attendance? Delay publishing of the recordings (just post them raw) by 12 weeks?
The collaboration mediums should scale with the number of folks in the population. The venue stays the same but you 10x the input, lots of great people and great research is going to get broken.
In AI/ML job ads it is quite typical for the requirements to include, "must have published at top AI conferences include but not limited to NeuroIPS, ICML, ICLR, ...etc", which I find completely crazy, because it just incentivizes grad students to publish rubbish papers to have on their CV, and indeed most conference papers in AI/ML are complete rubbish, because it is trivial to take any architecture and any corresponding benchmark, tweak the architecture slightly and publish a paper. Even dud results are published as "promising". It's just a complete shitshow, and as somebody who is in the field it feels as though you cannot even complain because people get offended. You're just supposed to keep spinning the hamster wheel without posing any hard questions.
Moreover, having published at top conferences does not prove that somebody is going to be a good ML Engineer. It just proves that somebody knows how to write a compelling conference paper, which is a completely different skill set altogether.
imo its a little more difficult to publish at these conferences using: "take any architecture and any corresponding benchmark, tweak the architecture slightly and publish a paper". at t2-t3 conferences ... sure.
Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors. CRYPTO, the annual "S-tier" conference in California, has already let people attend virtually online, and is considering its options for next year.
Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
The whole show up to conferences internationally to network and put attendace on your CV thing is also not great for people looking after children, among others.
In practice if you want discussion and citation for your cryptography paper, it has to go on IACR eprint at some point. Being published in CRYPTO is still a major endorsement, but not the way people actually get hold of a copy these days.
> Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors.
It seems to get a lot more attention now that people from a different type of country are getting affected.
> Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
Do you mean safe for individuals or a choice of venue?
In the UK (which is the country I know best) individuals are fine once they get a visa, but its not a safe choice in terms of planning because the granting of visas for people from certain countries is unpredictable (so people you expect to be able to attend might not be allowed to).
The legal question (that was settled by a recent case) was whether the word woman in existing legislation includes transwomen. The particular case was primarily whether transwomen could qualify for preference given to women for certain posts "on the boards of certain public authorities in Scotland".
The toilets things getting the publicity is more a matter of the media being obsessed with that aspect of it. Obviously there are implications for that, but also for many other things.
The EHRC [1] (Equality and Human Rights Commission) has issued guidance [2] that suggests public toilets could be affected. In practice I think life will go on as usual in any organisation that's not explicitly anti-trans, but there's a risk that someone will make a statement by suing a venue for sex discrimination.
As far as I know, the judgement affects much wider issues than just the women's "quota" on boardrooms in Scotland. It seems like a lot of major employers and their lawyers are trying to understand the implications and promising they'll have a update for everyone "soon".
The Supreme Court ruling will also ensure that males are not present in women's prisons, women's hospital wards, women's sports, domestic violence refuges for women, and many other spaces designated as being single-sex.
It also confirms that, in law, sexual orientation is defined in terms of sex. One of the intervenors in the case included lesbian groups who were concerned that legal recognition of the rights of lesbian women would be rendered meaningless if heterosexual males could simply identify as such.
As for bathrooms, they are one of the few facilities for which access is based on trust. The activists who insist that they're going to use opposite-sex bathrooms regardless of what the law says are confirming that they can't be trusted to respect boundaries and stay out. Which says a lot really.
I'm merely following the logic of your argument, that people who violate rules demonstrate untrustworthiness and a lack of respect for boundaries; you did not specify an exception for those people whose civil rights are being infringed by those rules.
Are you saying that it's fair to extend such an exception?
Pretty funny OOPSLA/SPLASH is the first example given because that's exactly the conference that came to mind when I clicked to read the article.
Honestly though, I find much better luck in the workshops. They don't really have the same reach as the main conference, and the specificity of the topics means that the reviewers are usually much more focused on content rather than checking boxes. They want to make a good workshop for the attendees, so it's far more important for workshops to focus on actual content rather than the resume building activities. The LIVE workshop for instance doesn't even really require a paper: https://2024.splashcon.org/home/live-2024#Call-for-Submissio...
Other workshops require just an extended abstract. Or maybe a short paper that doesn't have to be archived. I find these venues easier to get into, easier to present at, and easier to have a good discussion with the attendees.
The reason behind the citation style is to serve the automated parsing of citations by research information systems, which can then roll up all of your contributions to the field into a single score which determines one’s entire worth in academia.
The author really should have recognized this, as it serves his point about careerism and brownie points.
The idea that being forced into a citation style stifles innovation is hilarious, especially coming from a computer scientist - formal systems are all we do. It’s not so hard, is it? Use a citation manager and have them generated for you!
The main issue I see is that papers are actually becoming so focussed on form that they are now unreadable. People prefer reading my blog for my papers than reading the papers themselves. In fact I hear people telling me they understood the blog _better_. The whole academic writing shtick has become so obtuse that not only writing is cumbersome, but so is reading.
The other side of all this academic brownie points via papers (and doing reviews, which has become "brownie points for gatekeeping") is that most academic software is not only unmaintained, but actually unusable. They rarely even compile, and if they do, there is no --help, no good defaults, no README, and no way to maintain them. They are single-use software and their singular use is to write the paper. Any other use-case is almost frowned upon.
One of the worst parts of Academic software is that if you re-write it in a ways that's actually usable and extensible, you can't publish that -- it's not new ("research") work. And you will not only have to cite the person who wrote the first useless version forever, but they will claim they have done it if your tool actually takes off.
BTW, there are academics who don't follow this trend. I am glad that in my field (SAT), some of the best, e.g. Armin Biere and Randal Bryant are not like this at all. Their software is insanely nice and they fix bugs many-many years after release. Notice that they are also incredibly good engineers.
> In fact I hear people telling me they understood the blog _better_. The whole academic writing shtick has become so obtuse that not only writing is cumbersome, but so is reading.
Legible writing has little incentive. You can write in a simple down-to-earth manner and risk having someone objecting "Your work is isomorphic to X, done years ago." Your counter-objection, "Well, X is so incomprehensible that no one had any chance of understanding it", will fall flat. Better not risk it, write in an obtuse form and withhold the source code.
What’s stopping you from writing papers in the style of your blog posts (this is an actual question, not a ‘why don’t you just…’)
That reviewers tend to not like papers that aren't written like papers.
Reviewers.
Even small deviations from academic style get negative responses from reviewers.
there is only 1 implementation available for an algorithm on which i did part of my bs internship, and it uses a lot of harcoding. It of course makes benchmarking and extending it much, much harder if you want to change some things
examples here https://github.com/search?q=repo%3AIBM%2FFedMA%20hard&type=c...
these hard coded parts are not easily adjustable
Unfortunately academic resource doesn't seem to be focused on the hard problems of today. A fast algorithm is nice, but we need to be able to maintain code long term, understand code, and fix bugs. I have 15 million lines of code I'm supposed to maintain and I know many others reading this work on larger projects. There is no way one human can write that much software. There is no way we can afford to throw it away even if there is a better way (well we could, but based on the last time we did that the cost would be over 1 billion dollars and 8 years - not a useful investment since there is no reason to think anything will make those prices go down). We need to keep it working.
This is a shock to many of our leaders - who were writing 8 bit assembly to do similar things. They commonly did throw away all the work of the last version since it only took them a few months to rewrite it for the exact features they needed. (having experience because they wrote it just a year ago means the rewrite as much faster, and the limitations of 8 bit means it was worth rewriting since they had to remove one feature to add a new one).
Not to mention, academia likes to think it is isolated from the real world, which is advertising and marketing all the way down. We should probably mint more dual CS and marketing majors.
That's not a part of computer science that's organizational and process management at that point not computer science so I'm not shocked CS academics aren't interested in it. You're looking at the wrong group to answer your question, we have a great amount of discussion about how to organize large software to make it manageable already too but most of it comes down to breaking it into more manageable chunks with defined interfaces and testing that interface thoroughly.
Fir enough if the other group was looking at it but they are not from what I can tell.
ane frankly CS without those skills is useless for the reason I was replying to. If they cannot make readable/maintainable code they need to get a different job.
There's loads of space in a CS career where you'll never get anywhere near responsibility for a hundred thousand much less multiple millions of lines of code at once.
And there are already loads of advice out there about how to organize and manage huge code bases. It's usually consultancy based for anything but the broadest ideas though because it's so specific to the particular situation. An architecture is as much defined by the application it's built for as it is by what people would like to impose on it for convenience.
The article says that Attention is All You Need wasn't published in a "standard" conference, but wasn't it published in NIPS? Did I misread the article?
A big part of the problem here is that Universities have increasingly begun attaching prestige to specific “top” conference publications for both ranking and faculty promotions. A good example of the phenomenon can be seen in [1] (sorry for the noun-citation!) which only gives credit for approximately three conferences in each field. Combine this with a flood of new researchers entering CS, you have a recipe for “top” conferences being essentially destroyed and filled with uninspired work.
(And contrary to the joke in the article, even your own work becomes uninspired when you ship it to those conferences. You can’t afford to be quirky or interesting.)
Fortunately every field has a fourth or fifth-tier conference that isn’t on this list (or a specialized topic conference that the rankings folks don’t care about), and those still serve the purposes that conferences were made for. You just might not be able to convince a ranking-obsessed administrator that your work has any value if you publish there.
[1] https://csrankings.org/
(I imagine you agree, so this is just to expand) a secondary, insidious issue is that administrators diffuse their rules through the bureaucracy. In the case of CS, you start seeing references to csrankings in recommendation letters for grad applications, faculty applications, or even tenure letters. At that point, it can be hard to fight against it.
Note how the author talks about "main conferences", "major conferences", and "top conferences". That's the root issue. Whenever there is prestige available, people will compete for it. And if you have a competition, you should formalize the rules to make it fair.
When I was doing PhD ~15 years ago, I noticed that I rarely cited work that appeared in the top conferences of the subfield. Those conferences covered so wide range of topics that often only 1 or 2 papers were in the same subsubfield as me. And even those were often not directly relevant to my work.
But then there were small specialized conferences that had plenty of interesting papers every year. I left CS for another field a decade ago, but I still regularly attend some of those conferences and review for them. The papers published in them are still interesting and relevant to my work.
What are you doing now?
This article resonated with me and puts into words some of the feelings I had towards the end of my PhD. This part:
> An interesting case in software engineering is dismissal for lack of “evaluation.” It would be, of course, ridiculous to deny the benefits that the emphasis on systematic empirical measurement has brought to software engineering in the last three decades. But it has become difficult today to publish conceptual work not yet backed by systematic quantitative studies.
struck a chord with me. The top-tier CS systems conferences for me (OSDI and SOSP) have gotten to the point where you basically have to be writing the paper about the system you built at a FAANG that serves 1B users daily to get accepted.
It's hard for a novel idea and first-cut implementation to compete with systems built over many years with a team of a dozen software engineers. Obviously, those big systems deserve tons of credit and it's amazing that Big Tech publishes those papers! Credit to them. But it's also the case that novel ideas with an implementation that hasn't seen 1B users yet still have value.
I suppose the argument is that workshops serve that purpose of novel ideas with unproven implementation. There's some truth to that, but as the article highlights, the full conference papers are the real currency.
A great link nugget from the article: https://wp.doc.ic.ac.uk/cairesfe/wp-content/uploads/sites/80... - a parody on how some historically very significant papers might be rejected in today's system. made me chuckle a lot
I think the example of how to "correctly" cite a paper actually makes this issue seem smaller than it is. In reality, these conferences have very complicated (and unstated) "rules" for how a paper is supposed to look. If an "outsider" wanders in and submits a paper with new ideas, it will be very obvious that they are not a "member of the community" and their paper will usually be treated much more harshly as a result. This adds a huge amount of friction to research.
And what's particularly frustrating is that many organizers will try to combat this by writing papers saying they "particularly encourage" papers that are interdisciplinary, or focused on less fashionable topics, etc. It's good that they are trying to change things, but I think the main effect in practice is to encourage people to spend their time writing papers that have little chance of being accepted.
This issue isn't at all unique to computer science, though. Try publishing a paper in a top economics journal as an outsider!
I am fairly certain this rule was there against an obnoxious citing style of "The lambda calculus [1] was intended as a foundation for mathematics". It is especially obnoxious in the case of CS because when you cite e.g. "as Johns comments in his article about future developments of the programming languages [1963a]" it is quite important to know that this paper is actually from 1963 and can be mostly disregarded except as a historic curiosity; yet I've seen people vehemently defending this "[1]" style.
Is citation style really an issue? Even if they don't state which style they expect, surely you can tell their expected style from their existing publications? With proper tooling (e.g. LaTeX+BibTeX) it's pretty painless to switch styles.
Here's that rant of a blog from D.J. Bernstein [0] about how "[3, 7, 42]" citation style is superior and promotes scientific progress that I was thinking about when I wrote my comment. I personally find most of his reasoning pretty unconvincing; and so while I understand Meyer's irritation, I have to say I have to side with OOPSLA here. After all, you'd also probably want the submitted papers to be written in somewhat better than 5th-grade-high-school-student's English, and don't have way too many typos (I talk like ~15 typos per page).
[0] https://blog.cr.yp.to/20240612-bibkeys.html, previously discussed on HN here [1].
[1] https://news.ycombinator.com/item?id=40662056
Opinions vary on citation styles, my point was that it seems reasonable for a publication to standardise on one citation style, i.e. to require its use. I'm not sure it's what dynm meant by very complicated (and unstated) "rules" for how a paper is supposed to look. That article by DJB mentions that every author really ought to be using a citation-management solution like BibTeX, so that regardless of your preferences, it's easy to change your whole paper to a different citation style.
There's a slight (but only slight) irony in your use of the HackerNews convention for handling multiple links without breaking up the body of the main text. In this short-form medium it works great. I see someone made this same point in the thread you linked, at https://news.ycombinator.com/item?id=40673426
The only time I like numbers is writing proposals and I only like it because it saves space. Other than that I much prefer (name, year) if I am to have a preference at all.
Adding to the frustration, (the lack of) these shibboleths partially undermine double-blind reviewing, which is on the rise in prestigious conferences. A reviewer from the in-group may immediately spot that a submission comes from the out-group.
When I started my PhD in CS/AI in the late 90s, my department's AI faculty was already telling us, "at the major conferences, all the action is in the workshops." And this was my experience, the workshops were indeed where you found the most engaging experiences and interesting new things.
Meanwhile the work at the main at the main conference of AAAI or ICML was much farther along, and the value of having it presented at a conference, rather than a journal, was minimal. The conventional wisdom was, "the talk is just an advertisement for the paper."
There’s a larger problem here beyond careerization that is going unsaid. People are using these metrics as an assessment of their self worth. I’ve had top researchers point out that such and such publication was accepted in the industrial track vs research track, and how important it was to keep that flag in the bibliographical data, even well after publication in a top conference. Most papers research or industrial stay in obscurity. The truly novel ones will catch like wildfire. The metricization of research and academic standing is the underlying culprit.
> The truly novel ones will catch like wildfire.
This doesn't always happen; there are many historical examples where the first publication of an idea or technology goes unnoticed, often for many years, until someone better-positioned re-invents the same thing.
The publication of scientific papers is broken too:
1. authors that just reviewed the paper, did not do anything substential 2. papers that do not ship with working code 3. papers that are meaningless
Yes papers not having working code is very annoying and anti intellectual. I wrote a substack about this recently because I review a lot of papers that essentially are trash because the authors do not provide working codes:
https://open.substack.com/pub/mnky9800n/p/how-to-format-code...
1) Increase page length maximums, 10 pages is too low to explain something. I am looking directly at you IEEE.
2) Record and post all of the conferences, ACM is doing great strides here, but many many conferences aren't even recorded.
3) Reduce the paywalls and the access to all information, again I want to really support the ACM here, but the conferences that aren't broadcast for free (and also aren't recorded) are also the ones that are hella expensive and you have to attend in person (making them 5x as expensive with travel and lodging). Provide a happy medium, $300 for remote attendance? Delay publishing of the recordings (just post them raw) by 12 weeks?
The collaboration mediums should scale with the number of folks in the population. The venue stays the same but you 10x the input, lots of great people and great research is going to get broken.
In AI/ML job ads it is quite typical for the requirements to include, "must have published at top AI conferences include but not limited to NeuroIPS, ICML, ICLR, ...etc", which I find completely crazy, because it just incentivizes grad students to publish rubbish papers to have on their CV, and indeed most conference papers in AI/ML are complete rubbish, because it is trivial to take any architecture and any corresponding benchmark, tweak the architecture slightly and publish a paper. Even dud results are published as "promising". It's just a complete shitshow, and as somebody who is in the field it feels as though you cannot even complain because people get offended. You're just supposed to keep spinning the hamster wheel without posing any hard questions. Moreover, having published at top conferences does not prove that somebody is going to be a good ML Engineer. It just proves that somebody knows how to write a compelling conference paper, which is a completely different skill set altogether.
imo its a little more difficult to publish at these conferences using: "take any architecture and any corresponding benchmark, tweak the architecture slightly and publish a paper". at t2-t3 conferences ... sure.
Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors. CRYPTO, the annual "S-tier" conference in California, has already let people attend virtually online, and is considering its options for next year.
Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
The whole show up to conferences internationally to network and put attendace on your CV thing is also not great for people looking after children, among others.
In practice if you want discussion and citation for your cryptography paper, it has to go on IACR eprint at some point. Being published in CRYPTO is still a major endorsement, but not the way people actually get hold of a copy these days.
> Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors.
It seems to get a lot more attention now that people from a different type of country are getting affected.
> Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
Do you mean safe for individuals or a choice of venue?
In the UK (which is the country I know best) individuals are fine once they get a visa, but its not a safe choice in terms of planning because the granting of visas for people from certain countries is unpredictable (so people you expect to be able to attend might not be allowed to).
Safe for individuals, mainly.
The UK right now is also trying to figure out who can use what bathrooms. I don't understand the details myself.
The legal question (that was settled by a recent case) was whether the word woman in existing legislation includes transwomen. The particular case was primarily whether transwomen could qualify for preference given to women for certain posts "on the boards of certain public authorities in Scotland".
https://www.supremecourt.uk/cases/judgments/uksc-2024-0042
The toilets things getting the publicity is more a matter of the media being obsessed with that aspect of it. Obviously there are implications for that, but also for many other things.
The EHRC [1] (Equality and Human Rights Commission) has issued guidance [2] that suggests public toilets could be affected. In practice I think life will go on as usual in any organisation that's not explicitly anti-trans, but there's a risk that someone will make a statement by suing a venue for sex discrimination.
As far as I know, the judgement affects much wider issues than just the women's "quota" on boardrooms in Scotland. It seems like a lot of major employers and their lawyers are trying to understand the implications and promising they'll have a update for everyone "soon".
[1] https://www.gov.uk/government/organisations/equality-and-hum... [2] https://www.equalityhumanrights.com/media-centre/interim-upd...
Bathrooms are only a small part of this.
The Supreme Court ruling will also ensure that males are not present in women's prisons, women's hospital wards, women's sports, domestic violence refuges for women, and many other spaces designated as being single-sex.
It also confirms that, in law, sexual orientation is defined in terms of sex. One of the intervenors in the case included lesbian groups who were concerned that legal recognition of the rights of lesbian women would be rendered meaningless if heterosexual males could simply identify as such.
As for bathrooms, they are one of the few facilities for which access is based on trust. The activists who insist that they're going to use opposite-sex bathrooms regardless of what the law says are confirming that they can't be trusted to respect boundaries and stay out. Which says a lot really.
I'm not sure that it says what you think it says. It seems to equally apply to the trustworthiness of people like Rosa Parks.
How so? Are you suggesting that the civil rights of males are being violated when they're told that female-only spaces are off-limits?
I'm merely following the logic of your argument, that people who violate rules demonstrate untrustworthiness and a lack of respect for boundaries; you did not specify an exception for those people whose civil rights are being infringed by those rules.
Are you saying that it's fair to extend such an exception?
You're taking part of my comment and ignoring the context in which it was written.
Pretty funny OOPSLA/SPLASH is the first example given because that's exactly the conference that came to mind when I clicked to read the article.
Honestly though, I find much better luck in the workshops. They don't really have the same reach as the main conference, and the specificity of the topics means that the reviewers are usually much more focused on content rather than checking boxes. They want to make a good workshop for the attendees, so it's far more important for workshops to focus on actual content rather than the resume building activities. The LIVE workshop for instance doesn't even really require a paper: https://2024.splashcon.org/home/live-2024#Call-for-Submissio...
Other workshops require just an extended abstract. Or maybe a short paper that doesn't have to be archived. I find these venues easier to get into, easier to present at, and easier to have a good discussion with the attendees.
The reason behind the citation style is to serve the automated parsing of citations by research information systems, which can then roll up all of your contributions to the field into a single score which determines one’s entire worth in academia.
The author really should have recognized this, as it serves his point about careerism and brownie points.
The idea that being forced into a citation style stifles innovation is hilarious, especially coming from a computer scientist - formal systems are all we do. It’s not so hard, is it? Use a citation manager and have them generated for you!