Professor Amit Shesh

Payload Scanning for Volume Estimation and Visualization

This is an industry-funded project for which I am the sole PI. I have also developed a significant portion of the source code for this project. This project aims to create visualizations of and measures truck payloads using LiDAR scanning technology. Future customers in the mining and construction industries will use this application to examine payload distributions in the truck, and estimate their volumes. All measurements and calculations will be done on the fly as the truck drives under a scanning station. A wide variety of trucks (mining, dump, trucks with tarp supports, etc.) and types of payloads are supported by this system. This work will soon be part of commercial products.

Using graphics to teach Math

The foundations of computer graphics theory lie in mathematics, whereas most effects that add realism to a virtual graphical world are a direct simulation of the rules of physics. Can graphics be the motivation to learn Math and physics? This preliminary work illustrates this by constructing a virtual game world that de-constructs itself on command, revealing how its geometry and lighting was put together.

In future I wish to investigate if concepts used and taught in computer graphics can be directly related to topics in Math classes, so that students are motivated by visuals to learn and understand Math being taught to them.

Publications:

• Amit Shesh, Teaching Math and Physics by deconstructing graphics, Proc. GRAPP/IVAPP, pp. 523-526, 2012. [PDF]

Computer Graphics Education

If your department could offer just one graphics course, what would it contain? Which one-semester curriculum offers a good combination of theory, practical programming and fun, while relating to other courses in the program? Many universities offer only a single "stand-alone" course in computer graphics, due to insufficient room in the curriculum, limited faculty expertise and enrollment realities. This work documents my experience of teaching the computer graphics course at Illinois State University for 5 years, as I changed topics and course work to include more and more topics while keeping student motivation high. My students develop a scene graph, write shaders, create simple models and write a ray tracer all in one course.

In the future I would like to include topics from game programming, graphics on mobile devices in such courses. Data or scientific visualization is also on my wish list.

Publications:

• Amit Shesh, Towards a singleton undergraduate computer graphics course in small and medium-sized colleges, ACM Transactions on Computing Education 13(4), pp. 17:1-17:21, 2013. [PDF]
• Amit Shesh, Teaching Graphics to Students Struggling in Math: An Experience, Eurographics (Education Papers), 2015. [PDF]

Progressive Application Development in a single Programming Course

As students are exposed to more computing devices, how can programming be taught in an all-encompassing manner? Is it possible to emphasize programming concepts and yet develop a meaningful application from scratch in a single programming course? This work documents our experience of combining desktop applications, web services and Android programming in a programming course to progressively build a bus-routing application on Java and Android.

This work is representative of how I strive to include practical applications as course work, while expecting students to develop an application without a lot of start-up code given to them by the instructor.

Publications:

• Amit Shesh and Douglas P. Twitchell, Interplay of Desktop and Mobile apps with web services in an introductory programming course, Proc. AMCIS, 2013. [PDF]

Programming using Image Processing for non-Computer Science majors

In many ways it is a unique challenge to teach programming and high-level application development to beginners, especially non-computer science majors. This work shows how simple visual computing can help motivate students to learn more and be excited about programming. This work documents a semester-long experience of using image-processing as the theme in a course to teach programming and program design to students of information systems. Students progressively built a fairly complete image processing application from scratch in a bottom-up fashion using Java.

Publications:

• Amit Shesh, High-Level Application Development for non-Computer Science majors using Image Processing, Proc. Eurographics (Education Papers), pp. 37-41, 2011. [PDF]
• Amit Shesh, High-Level Application Development for non-Computer Science majors using Image Processing, Computers and Graphics 36(3), pp. 170-177, 2012. [PDF]

Automatic and semi-automatic tree model generation

The aim of this project is to create tree models and render them efficiently in a graphics scene. Although tree models are often big and complicated because of their structure, they are mere props in most scenes. Thus it is desirable that they be rendered in a scene without consuming a lot of computing resources. This project aims at creating a representation of mature trees that makes it easy to design them, and more importantly, render them efficiently. The vision of this project is that different tree models be simply dropped into a scene interactively without adversely affecting the frame rate.

Publications:

• Amit Shesh, GPU-generated "Parameterized" trees, Proc. Spring Conference on Computer Graphics (SCCG), pp. 147-152, 2010. [PDF]

Out-of-bounds Imagery

I participated in this project as a research intern at Microsoft Research Cambridge, working with Dr. Carsten Rother and Dr. Antonio Criminisi. The aim of this project is to be able to quickly create out-of-bounds images from ordinary photographs. While such effects can be created using Photoshop, they require considerable skill. Moreover Photoshop is not well suited to create out of bounds photographs because it provides very little support for prototyping (being able to quickly try out various OOB versions of the same photograph). We use basic computer vision to create a user-interface that allows 2D image manipulations in a "3D-correct" manner, thereby ensuring correct 3D effects without the user having to perform 3D manipulations.

Publications:

• Amit Shesh, Antonio Criminisi, Carsten Rother and Gavin Smyth, 3D-aware image editing for Out-of-bounds Photograph, Proc. Graphics Interface, pp. 47-54, 2009. [PDF]

Sketch-based Geometry Reconstruction

The aim of this project, entitled SMARTPAPER, is to convert 2D rough, hand-drawn sketches of solid objects into 3D geometry, using minimal and intuitive user interaction. We concentrate on developing core algorithms for the interactive reconstruction of 3D geometry without any prior knowledge about the nature of solid objects being sketched. Using SMARTPAPER on a Tablet PC, the user can sketch exactly as on paper, and even use intuitive gestures to aid in the reconstruction process. Sketch-based reconstruction of 3D geometry finds applications in the early stages of mechanical engineering and architectural design.

Publications:

• Amit Shesh and Baoquan Chen, SMARTPAPER--An Interactive and Easy-to-use Sketching System, Computer Graphics Forum 23(3) (Proc. Eurographics), pp. 301-310, 2004. [PDF]
• Amit Shesh and Baoquan Chen, User Interface Design and Realization of a Design-by-Sketches System, Technical Report (TR.04.01), Department of Computer Science, University of Minnesota at Twin Cities, 2004.

Sketch-based Photo Navigation

The aim of this project, entitled Peek-in-the-Pic, is to impart a third dimension to photographs, so that it is possible to fly through them. Our system offers an easy interface to interactively construct 3D geometry starting from a single, casually taken photograph of architectural objects like buildings. Using our system, the user can virtually ``fly through'' a photograph to get a feeling of ``being there'', even without having actually visited the place (virtual photo tourism).

Publications:

• Amit Shesh and Baoquan Chen, Peek-in-the-Pic: Architectural Scene Navigation from a Single Picture Using Line Drawing Cues, Pacific Graphics 2005 (Short paper)[PDF]
• Amit Shesh and Baoquan Chen, Peek-in-Peek: Flying Through Architectural Scenes From a Single Image, Computer Graphics Forum 27(8) pp. 2143-2153, 2008.[PDF]

Sketch-based Inverse Lighting

The aim of this project, entitled Crayon Lighting, is to provide a sketch-based interaction metaphor for a completely goal-based way of designing lights for a particular scene. A user roughly sketches desired lighting effects like highlights and shadows in a given scene, from which the system determines where to place lights to produce these effects. Such a system finds applications in designing lights for buildings and in animations to produce desired cinematic effects.

Publications:

• Amit Shesh and Baoquan Chen, Crayon Lighting: Sketch-guided Illumination of Models , ACM GRAPHITE, pp. 95-102, 2007. [PDF]
• Amit Shesh and Baoquan Chen, Crayon Lighting: Sketch-based Illumination of Models, Pacific Graphics(Poster paper), 2006[PDF]

Rendering line models with fidelity

The style and abstract nature of hand-drawn sketches is important in many design and artistic applications, due to which it is undesirable to refine, beautify or even parse them. The aim of this project is to retain the original nature of hand-drawn sketches and concentrate on rendering them automatically and faithfully. We address several fundamental challenges of rendering 2D/3D wire-frame sketches, propose and implement a rendering pipeline based on lines and curves as basic rendering primitives. Not only does this find important applications in sketch-based art and design, but also in more efficient data representations for non-photorealistic rendering of geometry.

Publications:

• Amit Shesh and Baoquan Chen, Efficient and Dynamic Simplification of Line Drawings, Computer Graphics Forum 27(2) (Proc. Eurographics), pp. 537-545, 2008.[PDF]

Sketch-based Segmentation of Point Models

When modeling with scanned outdoor models, being able to select a subset of the points efficiently that collectively represent an object is an important operation. We propose a sketch-based interface that allows such segmentation. The user marks object and background regions by placing strokes, and the tool segments out the marked object(s). To refine the results, the user simply moves the camera to a different location and repeats the process. Our method works for general 3D point models and not just range images.

Publications:

• Xiaoru Yuan, Hui Xu, Minh X. Nguyen, Amit Shesh and Baoquan Chen, Sketch-based Segmentation of Scanned Outdoor Environment Models, Proc. Eurographics Symposium on Sketch-based Interfaces and Modeling (SBM), 2005.[PDF]

List of Publications

Doctoral dissertation:,Using Hand-drawn Sketches for Modeling and Rendering in Computer-based Design and Art, Advisor: Dr. Baoquan Chen, University of Minnesota (2008) Thesis

Journal Publications

1. Amit Shesh,Toward a Singleton Undergraduate Computer Graphics Course in Small and Medium-sized Colleges, ACM Transactions on Computing Education (TOCE) 13.4, pp. 17:1-17:21 (2013). Paper.
2. Amit Shesh,High-level application development for non-computer science majors using image processing, Computers & Graphics 36.3, pp. 170-177 (2012). Paper.
3. Amit Shesh and Baoquan Chen, Efficient and Dynamic Simplification of Line Drawings, Computer Graphics Forum (Proc. Eurographics) 27.2, pp. 537-545 (2008). Paper.
4. Amit Shesh and Baoquan Chen, Peek-in-the-Pic: Flying Through Architectural Scenes From A Single Image, Computer Graphics Forum 27.8, pp. 2143-2153 (2008). Paper.
5. Amit Shesh and Baoquan Chen, SMARTPAPER: An Interactive and User Friendly Sketching System, Computer Graphics Forum (Proc. Eurographics) 23.3, pp. 301-310 (2004). Paper.

Peer-reviewed Conference papers and technical reports

1. Amit Shesh, Teaching Graphics to Students Struggling in Math: An Experience, Eurographics (Education Papers), 2015
2. Amit Shesh and Douglas P. Twitchell, Interplay of Desktop and Mobile Apps with Web Services in an Introductory Programming Course, Proc. AMCIS 2013. Paper.
3. Matthew Jones and Amit Shesh, Scene graph Creation and Management for Undergraduates, Proc. Eurographics Teaching Papers (Outstanding student projects), pp. 11-12 (2013) Paper.
4. Amit Shesh, Teaching Math and Physics by Deconstructing Graphics, Proc. GRAPP/IVAPP., pp. 523-526 (2012) Paper.
5. Amit Shesh, High-Level Application Development for non-Computer Science majors using Image Processing, Proc. Eurographics (Education Papers), pp. 37-41 (2011). Paper.
6. Amit Shesh, GPU-generated Parameterized Trees, Proc. Spring Conference on Computer Graphics (SCCG), pp. 147-152 (2010). Paper.
7. Amit Shesh, Antonio Criminisi, Carsten Rother and Gavin Smyth, 3D-aware Image Editing for Out Of Bounds Photography, Proc. Graphics Interface, pp. 47-54 (2009). Paper.
8. Amit Shesh and Baoquan Chen, Crayon Lighting: Sketch-guided Illumination of Models, Proc. ACM GRAPHITE, pp. 95-102 (2007). Paper.
9. Amit Shesh and Baoquan Chen, Crayon Lighting: Sketch-based Illumination of Models, Proc. Pacific Graphics (Poster paper) (2006). Paper
Amit Shesh and Baoquan Chen, Lasels--Line-based Primitives for Representing and Rendering of 2D and 3D Sketches with Fidelity, Technical report University of Minnesota (TR.06.04) (2006).
10. Amit Shesh and Baoquan Chen, Peek-In-The-Pic: Architectural Scene Navigation from a Single Picture Using Line Drawing Cues, Proc. Pacific Graphics (Short paper) (2005). Paper
11. Xiaoru Yuan, Hui Xu, Minh X. Nguyen, Amit Shesh and Baoquan Chen, Sketch-based Segmentation of Scanned Outdoor Environment Models, Proc. Eurographics Symposium of the Sketch-based Interfaces and Modeling, pp. 19-26 (2005) Paper.
12. Clement Shimizu, Amit Shesh and Baoquan Chen, Hardware Accelerated Motion Blur Generation, Technical report University of Minnesota (TR.03.05) (2003).