World Cup 2022 | Fox Sports
During the Fall Quarter of 2019 I was selected to work on a SCADpro project with Fox Sports. The premise of the project was to create a 3D studio set for broadcasters of the Fifa 2022 event in Qatar. This virtual space was made to be incorporated into Fox sports 360 degree green screen studio. Along with 11 other SCAD students,we were tasked with creating an environment that represented Qatar and Fox Sports,was interactive, and gave way to visuals and interactions that would be too expensive or impossible to reproduce in real life.
I was in charge of the AR or MR (Mixed Reality) of the project. I was given a Leap Motion which I connected to Unreal Engine in order to create interactions in engine based off of human gesture control. This was my first experience in Unreal with external equipment, so a lot of research went into learning how to approach this task.
The main interactive element I created was the interactive pool; It serves as a decorative element in the environment until it is used by a talent (broadcaster). The pool can be activated with gesture controls; which converts the pool into a soccer field on which talents can discuss player formations, attack strategies and more. The field can also be used to display Logos, team and game information, ect.
I was also in charge of other miscellaneous interactions including material transitions on the tapestries and walls, the animation of a dome which covers the ceiling, and an interactive Soccer Ball at the center of the room; serving as the hero asset for the environment. The Soccer ball can change its segments into screens to display team flags, information, or serve as a scoreboard.
AR Pool Process
The entire project only lasted ten weeks; at first the team was split into three groups which created pitches for our client. We had not settled on the idea of making an interactive pool yet and my group had pitched using a 3D Soccer ball as a main AR/interactive asset. The Leap Motion tool was given to me, and I began working on the soccer ball, with only a few weeks to familiarize myself with the Leap Motion hardware. The Leap Motion is a tool that tracks hand motion; which allowed me to create specific hand gestures and signals that can be read in Unreal Engine. Soon I had created a system that allowed an item to be moved, scaled and rotated all through gesture control.
Testing the Leap Motion with Unreal Engine
Functional scale and rotation
based of gesture control
Progression of the Reflection Pool System
As we met with the client and the other groups we oriented towards using the pool as an interactive asset as it served as a better interactive surface; considering the camera angles and the density of the environment. I was then tasked with coming up with creating different inputs that could be controlled by gestures, which had to be simplistic and minimal; as all of gestures would be seen on live television. I decided to only use directional hand swipes and pinches for gestural inputs. I had an idea to give the talent a physical podium which held a screen. I developed a split screen feature for the project; one screen would show all of the possible options and selections for setting up the next gesture. This screen would guide the user through gestures, making the process more user friendly. The second screen shows a live audience view; including our virtual set with the broadcasters.
AR Pool Full Functionality
With the tactile screen serving as a control board; the main use out of this interactive pool would be to show data and information in an animated, virtual environment; something more interesting than static UI. The screen would have selections and based on that a specific gesture would be required to activate an action. One of these actions would involve swinging your hand up in order to pull a logo, such as the Fox Sports logo or team logos out of the pool. The most important feature is the ability to open up the pool and change it into a soccer field on which a full 3D soccer team would spawn, complete with 3D AI controlled players. If the talent chose a different team or formation the players will change dynamically and the 3D players would actually run to their new formation position. You could even simulate the way a team would form an attack and see it happen real time. Using a simple pinch gesture the talent can command the virtual camera to zoom in on one of these players, and show all of his relevant stats. The soccer player can also play specific animations in these moments. This system serves as a proof of concept for something that could control many other elements; including the virtual set,a day and night cycle, activating different screens in the environment, or controlling cameras in a seamless way for live television.
Interactive Soccer Ball
The idea behind this interactive ball is that there are 32 teams that participate in the Fifa World Cup, and 32 panels on a soccer ball. Conveniently, we could have the flag of each country displayed on each panel of the ball. We planned for the ball to be in the center of the room, and react accordingly to any hand gestures. The original idea was for the ball's panels to be able to move in order to display screens, either by detaching and enlarging, or potentially retracting and creating a hologram (See previs GIFS).
The project did not follow this direction and the ball ended up serving as a display; showing the score, team info, or any other subjects. I created a dynamic material that could be modified in order to show a countries flag, or the current score of the game.
Previs idea: Hologram
Previs idea: Interactive Screen