11 points | by accheng 9 hours ago
10 comments
We modelled using Redshift scheduling automation, that suggested a welcome 20% improvement.
We applied it and it run 20% slower.
As it runs it is supposed to learn how to optimise scheduling. Not sure how it gets over the burst at month, quarter, and year end.
What did you use to try to optimize scheduling?
Transactions seem like a hard problem for scheduling, wonder how the authors got this to work in the original paper?
As described in our VLDB paper (https://www.vldb.org/pvldb/vol17/p2694-cheng.pdf), we had to implement a number of optimizations and integrate scheduling with CC to get good performance.
For the offline problem, AI found a new solution that was 34% faster. Interesting results!
I wonder what the optimal for these workloads are?
What about other use cases
Check out our blog series here: ucbskyadrs.github.io!
Seems like AI hype
turns out DB transactions are a hard problem...
We modelled using Redshift scheduling automation, that suggested a welcome 20% improvement.
We applied it and it run 20% slower.
As it runs it is supposed to learn how to optimise scheduling. Not sure how it gets over the burst at month, quarter, and year end.
What did you use to try to optimize scheduling?
Transactions seem like a hard problem for scheduling, wonder how the authors got this to work in the original paper?
As described in our VLDB paper (https://www.vldb.org/pvldb/vol17/p2694-cheng.pdf), we had to implement a number of optimizations and integrate scheduling with CC to get good performance.
For the offline problem, AI found a new solution that was 34% faster. Interesting results!
I wonder what the optimal for these workloads are?
What about other use cases
Check out our blog series here: ucbskyadrs.github.io!
Seems like AI hype
turns out DB transactions are a hard problem...