Summary and Contributions: The paper presents algorithms for intersection control that can cope with the scarcity of compute resources and network bandwidth in developing countries. Instead of using previously published heavy Deep RL algorithms, the paper presents results for 1 or 2 dimensional RL, and then replaces the RL run by using lookup tables. It next presents a stateless algorithm for the case of RAM constraints, when the lookup table cannot be used. The evaluation of the algorithms is done both on public NYC data, considered in previous papers, and on data collected in one intersection in a developing country. That dataset had the additional challenge of not having lanes (the traffic density was analyzed using CNN and optical flow). The evaluation results show performance and fairness in the ballpark of the heavier algorithms. In some cases, the simpler algorithms presented in the paper have an advantage over the previous heavier ones.
Strengths: This paper considers both an interesting use-case that has a potential impact on society, and an interesting algorithmic problem. It presents non-trivial solutions, which were demonstrated in an actual deployment (a demo). The solutions address both the RL aspect of finding the right traffic lights policy and the computer vision aspect of analyzing the traffic density without lanes. I think this could be interesting for the NeurIPS audience.
Weaknesses: The paper could benefit from showing additional results for implementation in developing countries, which is the motivation for the algorithmic work. Currently results are provided for a single demo in a single intersection. Also, the data of that intersection is not provided (the authors state that it will be provided upon acceptance). The computer-vision methods used for analyzing the images are not described in detail, their code is not provided, and there is no analysis of the correctness of this specific part (just an end-to-end analysis). If this is not done well, it could make all the algorithms become similarly weak.
Correctness: The claims and methods seem correct, although there are some weaknesses, as described in the previous paragraph.
Clarity: The paper is well written.
Relation to Prior Work: Prior work was clearly discussed, including differences from the current work.
Reproducibility: No
Additional Feedback:
Summary and Contributions: This paper studies intersection control under extremely low resource scenarios. The major contribution in my opinion is a (pilot phase) deployed end-to-end intersection control system that uses very limited computing resources. The main techniques being used are a combination of low-resource reinforcement learning and computer vision (to extract the input state end-to-end).
Strengths: 1. The system is deployed (pilot phase) in one intersection of a developing country. This has the potential of benefitting the society by exploiting the power of AI. 2. The problem setting where only extremely low resources are accessible is well motivated. The solution being given is well engineered and suits the target scenario. 3. The performances of the 3 major designs, namely state space reduction, off-line training and online execution and threshold-based policy have been demonstrated by simulation results. 4. The system also shows good robustness and fairness performances.
Weaknesses: 1. My main concern is that the paper, while being practical and useful in real-world, has limited technical contributions from an AI perspective. I do appreciate that the authors are trying to make the writing clear and simple, it somehow appears to be more like a collection of engineering efforts to make the system work. The paper is also written in a way that looks like a "technical report" rather than a machine learning type of paper. For example the Markov Decision Process underlying the problem is never formally defined, especially state transitions. It will be helpful to write down these definitions. 2. While the 2-d and 1-d state representation seems to work, there is no discussion as to "why" it works. What is the intuition behind this? Is it only restricted to the very simple settings (1*1 1*16 and 3*16) or is it generalizable to more complex problem scenarios? I would suggest the authors make a more in-depth discussion from this perspective.
Correctness: Yes.
Clarity: The paper is clearly written and easy to follow. Perhaps it could be presented in a more formal way with definitions of problem, and mathematical representations.
Relation to Prior Work: Yes.
Reproducibility: Yes
Additional Feedback: Thanks to the authors for clarifying my concerns in the rebuttal. My consideration of leaning toward rejecting the paper was based on the lack of technical contribution from an AI/ML perspective. Combing the rebuttal and other reviews I am convinced that it is an interesting direction for RL-based intersection control, and as mentioned by R5 and the rebuttal, is a promising field to "jump out of the pure application of traffic signal control problem". Therefore I will raise my score from 5 to 6. It will also be good for the authors to add a brief but formal description of the mathematic formulation of the problem.
Summary and Contributions: This paper tackles the traffic signal control problem with the consideration of infrastructural limits in developed countries. They tried the following: 1. simplifying the state representations. 2. look-up table methods (RL might not be supported due to infrastructure budgets). 3.Threshold-based methods (lower budget than look-up table methods). Overall the paper is well written, with considerations well-explained, methods reasonably adopted.
Strengths: 1. Sound empirical evaluation. This paper uses all open-source data and makes the experiments easier to reproduce. Secondly, this paper compared with plenty of baselines, including RL-based and traditional transportation methods, and the results seem to match the state-of-the-art performance. 2. Simple methods with clear motivations. The recent advances in RL-based methods seem to overcomplicate, with different tricks proposed yet hard to reproduce for low-budget communities. This paper shows a clear message: sometimes we might not necessarily complicate the problem and simple methods like look-up tables can also beat state-of-the-art RL methods.
Weaknesses: 1. The techniques in this paper require manual design that might not of the current NeurIPS community. Currently, the community seems to prefer automated learning-based/optimization-based methods. Yet the look-up table and threshold methods proposed in this paper might not be appreciated by some researchers.
Correctness: Yes
Clarity: Yes
Relation to Prior Work: Yes
Reproducibility: Yes
Additional Feedback: I have a few suggestions that might make this paper more convincing: 1. Jump out of the pure application of traffic signal control problem. It is preferable to see how the message "simpler methods beats complicates methods with budget constraints" generalize to other problems, especially real-world applications like transportation or health care (other than games or robotics control). The community would much like to see how in the real-world cases the complicated methods fail generally and how to mitigate the failure.