摘要:图像处理是图像生成领域的一种应用场景,其中生成图像是对原始图像的修改。在大多数情况下,图像生成和处理任务是在原始像素上进行操作。但是,学习丰富图像和目标表示两方面取得的显著进展为文本到图像或布局到图像等主要由语义驱动的任务开辟了路径。 在本文中,来自慕尼黑工业大学、牛津大学、约翰霍普金斯大学和谷歌的研究者基于场景图(scene graph)来解决图像生成新问题,其中用户仅通过应用图像生成语义图的节点或边缘改变,即可以编辑图像。研究目的是在给定的群集中对图像信息进行编码,进而生成新的群集,如目标替换以及目标之间关系的变化,同时原始图像的语义和风格保持不变。他们提出的空间语义场景图网络不需要直接监督群集变化或图像编辑,这使得人们可以从已有真实世界数据集中训练系统并且不需要做额外注释。 训练策略图示。 本研究方法(图中)与基线方法(图上)的视觉特征编码效果对比,其中场景图保持不变。 效果展示 3 个示例:骑马变牵马、树的位置从后方变前方,以及摩托车上变摩托车旁。 推荐:本研究提出的方法可以使用户在保持场景不变的情况下实现目标的位置变化。 ArXiv Weekly Radiostation 机器之心联合由楚航、罗若天发起的ArXiv Weekly Radiostation,在 7 Papers 的基础上,精选本周更多重要论文,包括NLP、CV、ML领域各10篇精选,并提供音频形式的论文摘要简介,详情如下: 10 NLP Papers.mp3来自机器之心00:0018:26 本周 10 篇 NLP 精选论文是: 1. M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training. (from Haoyang Huang, Lin Su, Di Qi, Nan Duan, Edward Cui, Taroon Bharti, Lei Zhang, Lijuan Wang, Jianfeng Gao, Bei Liu, Jianlong Fu, Dongdong Zhang, Xin Liu, Ming Zhou)2. Situated and Interactive Multimodal Conversations. (from Seungwhan Moon, Satwik Kottur, Paul A. Crook, Ankita De, Shivani Poddar, Theodore Levin, David Whitney, Daniel Difranco, Ahmad Beirami, Eunjoon Cho, Rajen Subba, Alborz Geramifard)3. A Survey of Neural Networks and Formal Languages. (from Joshua Ackerman, George Cybenko)4. A Unified Dual-view Model for Review Summarization and Sentiment Classification with Inconsistency Loss. (from Hou Pong Chan, Wang Chen, Irwin King)5. Extracting COVID-19 Events from Twitter. (from Shi Zong, Ashutosh Baheti, Wei Xu, Alan Ritter)6. Emergent Multi-Agent Communication in the Deep Learning Era. (from Angeliki Lazaridou, Marco Baroni)7. Response to LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts. (from Hao Wu, Gareth J. F. Jones, Francois Pitie)8. Syntactic Search by Example. (from Micah Shlain, Hillel Taub-Tabib, Shoval Sadde, Yoav Goldberg)9. Context-based Transformer Models for Answer Sentence Selection. (from Ivano Lauriola, Alessandro Moschitti)10. The Importance of Suppressing Domain Style in Authorship Analysis. (from Sebastian Bischoff, Niklas Deckers, Marcel Schliebs, Ben Thies, Matthias Hagen, Efstathios Stamatatos, Benno Stein, Martin Potthast) 10 CV Papers.mp3来自机器之心00:0021:55 本周 10 篇 CV 精选论文是:1. Look Locally Infer Globally: A Generalizable Face Anti-Spoofing Approach. (from Debayan Deb, Anil K. Jain)2. Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining. (fromYiqun Mei, Yuchen Fan, Yuqian Zhou, Lichao Huang, Thomas S. Huang, Humphrey Shi)3. UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content. (from Zhengzhong Tu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, Alan C. Bovik)4. Flexible Bayesian Modelling for Nonlinear Image Registration. (from Mikael Brudfors, Yaël Balbastre, Guillaume Flandin, Parashkev Nachev, John Ashburner)5. Recapture as You Want. (from Chen Gao, Si Liu, Ran He, Shuicheng Yan, Bo Li)6. DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution. (from Siyuan Qiao, Liang-Chieh Chen, Alan Yuille)7. HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens. (from Zhaohui Yang, Yunhe Wang, Dacheng Tao, Xinghao Chen, Jianyuan Guo, Chunjing Xu, Chao Xu, Chang Xu)8. Boundary-assisted Region Proposal Networks for Nucleus Segmentation. (from Shengcong Chen, Changxing Ding, Dacheng Taoo)9. CircleNet: Anchor-free Detection with Circle Representation. (from Haichun Yang, Ruining Deng, Yuzhe Lu, Zheyu Zhu, Ye Chen, Joseph T. Roland, Le Lu, Bennett A. Landman, Agnes B. Fogo, Yuankai Huo)10. Nested Scale Editing for Conditional Image Synthesis. (from Lingzhi Zhang, Jiancong Wang, Yinshuang Xu, Jie Min, Tarmily Wen, James C. Gee, Jianbo Shi) 10 ML Papers.mp3来自机器之心00:0020:02 本周 10 篇 ML 精选论文是:1. Learning Kernel Tests Without Data Splitting. (from Jonas M. Kübler, Wittawat Jitkrittum, Bernhard Schölkopf, Krikamol Muandet)2. Rethinking Empirical Evaluation of Adversarial Robustness Using First-Order Attack Methods. (from Kyungmi Lee, Anantha P. Chandrakasan)3. Learning Robust Decision Policies from Observational Data. (from Muhammad Osama, Dave Zachariah, Peter Stoica)4. DC-NAS: Divide-and-Conquer Neural Architecture Search. (from Yunhe Wang, Yixing Xu, Dacheng Tao)5. The Value-Improvement Path: Towards Better Representations for Reinforcement Learning. (from Will Dabney, André Barreto, Mark Rowland, Robert Dadashi, John Quan, Marc G. Bellemare, David Silver)6. Shapley Value as Principled Metric for Structured Network Pruning. (from Marco Ancona, Cengiz Öztireli, Markus Gross)7. Hierarchical forecast reconciliation with machine learning. (from Evangelos Spiliotis, Mahdi Abolghasemi, Rob J Hyndman, Fotios Petropoulos, Vassilios Assimakopoulos)8. Anomaly Detection with Tensor Networks. (from Jinhui Wang, Chase Roberts, Guifre Vidal, Stefan Leichenauer)9. The Convolution Exponential and Generalized Sylvester Flows. (from Emiel Hoogeboom, Victor Garcia Satorras, Jakub M. Tomczak, Max Welling)10. DeepCoDA: personalized interpretability for compositional health. (from Thomas P. Quinn, Dang Nguyen, Santu Rana, Sunil Gupta, Svetha Venkatesh)