Skip to content

Deep Learning Workshop (Spring 2022)

Friday May 6th, 2022 at Galanti Lounge, Robert L. Carothers Library (URI Kingston Map).

Placeholder Placeholder Placeholder Placeholder Placeholder


This workshop is being sponsored by a TensorFlow and Google AI award to support machine learning courses and diversity programs.


1-2p, Hierarchical Text-Conditional Image Generation with CLIP Latents

Aditya Ramesh [web], OpenAI [web]


Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We show that explicitly generating image representations improves image diversity with minimal loss in photorealism and caption similarity. Our decoders conditioned on image representations can also produce variations of an image that preserve both its semantics and style, while varying the non-essential details absent from the image representation. Moreover, the joint embedding space of CLIP enables language-guided image manipulations in a zero-shot fashion. We use diffusion models for the decoder and experiment with both autoregressive and diffusion models for the prior, finding that the latter are computationally more efficient and produce higher-quality samples.

Short Bio

Aditya Ramesh is a research scientist at OpenAI who works on generative modeling.

2-3p, An Introduction to Deloitte's AI Center of Excellence

Edward Bowen, Deloitte [web]

Short Bio

Ed Bowen is the Deloitte Advisory AI Leader and the Managing Director of the AI Center of Excellence. AI CoE Researchers and Engineers are commercializing novel AI models in areas like Cybersecurity, Fraud, Financial Controls, and Predictive Maintenance. Ed comes to Deloitte with approximately 20 years’ experience in Life Sciences, most recently as the Vice President of AI & Machine Learning at GlaxoSmithKline (GSK) where he led a large team of machine learning engineers using AI in the discovery and development of transformational medicines for patients. His team successfully delivered models across a wide range of medicines discovery problems including target validation, enhanced clinical trial design, protein engineering, and molecular design. Ed holds an MA in Biology from Brown University, an MBA from Rensselaer Polytechnic Institute, a Master of Science in Electrical Engineering from the University of Rhode Island and a Bachelor of Science in Electrical Engineering from the US Coast Guard Academy. He is currently completing a PhD in Molecular Biology at the University of Miami.

3-4p, Student's Poster Session and Coffee Break

  • AirDrums: Advanced IMU Recognition Driven by RNN in Unimpeded Multivariate Space (Matthew McAdams)
  • Deep Learning for Computational Fluid Dynamics (Pedro Mesquita)
  • Deep Learning for Motor Imagery Classification using Electroencephalography (James McIntyre and John McLinden)
  • Drug-drug interaction extraction using relation bidirectional encoder representations from transformers with bidirectional long-short term memory (Maryam KafiKang)
  • Finite Element Analysis and Convolutional Neural Network to Evaluate the Auxetic Response of 3D Printed Re-Entrant Metamaterials (Bolaji Oladipo)
  • General Policy Optimization for Discrete Action Environments (Nicholas Clavette)
  • Generating Text, Children’s Book Descriptions, with LSTMs (Emmely Trejo Alvarez and Lily Sisouvong)
  • Implementation and Enhancement of Deep Template-based Object Instance Detection (Travis Frink)
  • Improving Optimizing Compilers with Deep Reinforcement Learning (Raymond Turrisi)
  • Learning Compiler Optimizations using Transformers (Javier Vela)
  • Mini Cable-Driven Robotic Arm Range of Motion Process Failure Prediction and Root Cause Analysis using Machine Learning (Liyuan Gong)
  • Multivariate Time-series Forecasting on Stock Market Data (Alfred Timperley)
  • Musical Key Estimation Using Convolutional Neural Networks (Christopher Woyak)
  • Object Detection and Bounding Box Regression (Shuang Wang and Erkan Karakus)
  • Online Marketplace Tracking through Image Anomaly Detection (Daniel Marasco)
  • Swarm Classification with Neural Networks (Justin Maio)
  • Using GPT-2 and Deep Reinforcement Learning from Human Feedback For Academic Topic Recommendation (Brennan Richards)

4-5:15p, Panel: Career Paths in AI

Daniel Crispell, Vision Systems Inc. [web]

Short Bio

Dr. Crispell received his BS from Northeastern University in 2003 and Ph.D. from Brown University in 2010 where he focused on probabilistic 3D reconstruction algorithms using volumetric geometry representations. After 1.5 years as a visiting scientist at the National Geospatial Intelligence Agency (NGA), Dan co-founded Vision Systems, Inc. in 2011. Dan has published several research papers in the fields of 3D reconstruction, image and video registration, and change detection. He has developed and applied 3D reconstruction and automatic change detection processing techniques to overhead imagery and developed automatic fusion and exploitation algorithms for LIDAR, image, and video data. Current areas of investigation include semantic segmentation and representation learning for satellite imagery, 3D digital surface model (DSM) generation, and 3D spacecraft reconstruction and analysis from on-orbit imagery.

Arturo Deza [web], CBMM-MIT [web]

Short Bio

Arturo Deza received his B.Sc. in Robotics (Ingeniería Mecatrónica) in 2012 from Universidad Nacional de Ingeniería in Lima, Peru. He then completed his Ph.D. in Dynamical Neuroscience at the University of California, Santa Barbara in 2018 advised by Miguel Eckstein where he began his research on visual search and foveated vision in humans and machines. In 2019, Arturo moved to Harvard University as a PostDoctoral Fellow with Talia Konkle at the Department of Psychology, and from early 2020 to date he is a PostDoctoral Research Associate working with Tomaso Poggio at MIT’s Center for Brains, Minds and Machines. His research interests span across the fields of psychophysics, representation learning, human-machine perception, and robotics. Easter-Egg: Influenced by his surfing and his research, Arturo is also an artist and he has had his first solo exhibition in Massachusetts's Gallery in the Summer of 2021. Some of his works are on view at research labs in Harvard + MIT.

Kevin Bello [web], UChicago [web] / CMU [web]

Short Bio

Kevin is a Computing Innovation Fellow and postdoctoral researcher at the University of Chicago and Carnegie Mellon University working with Bryon Aragam and Pradeep Ravikumar. His research focuses on developing principled algorithms that are computationally and statistically efficient for different machine learning problems. His current interests include causal discovery and learning invariant/causal representations. Previously, Kevin received his Ph.D. in Computer Science from Purdue University, where he was advised by Jean Honorio. Before his Ph.D., Kevin received a BSc in Mechatronics Engineering from Universidad Nacional de Ingeniería in Lima, Peru.

Barton Fiske , NVIDIA