Three out of four reviewers believe that the paper should be accepted due to the interesting and new application domain, and a reasonable approach. The fourth reviewer is also positive, though she/he believes that the paper could be improved before acceptance. The rebuttal as well as the parts of supmat seem to address part of the concerns of this reviewer, and the authors are encouraged further improve the final version bringing in some results from the supmat into the main text (e.g. comparison with SVG-VAE). On balance the recommendation of the area chair is to accept.