This is one of the 360° video scenes in the Turku University of Applied Sciences’ virtual, interactive recovery room simulation. All the videos contain different phases, symptoms and statuses of patient care, that the nursing student has to interpret when practising perioperative care.
In this case, the 360° videos are built around a patient recovering from an appendectomy, but they can be applied to any learning situation. The recovery room case is just a practising tool to develop further the 360ViSi Editor constructed by Turku UAS staff as a part of the 360ViSi project.
Benefits of virtual simulation
Unlike a face-to-face simulation, a virtual one has numerous benefits. It allows the student to practice in real-like surroundings regardless of the time, place and schedules of other students and staff members. One part of the learning experience is to answer situation-related quizzes and to react to a patient’s symptoms. Answers and reactions gather data that enables giving individual and immediate feedback to the student.
Testing – a vital part of developing
At one of the Turku UAS’ meeting rooms, the nursing teachers tested the editor for the first time. As the Editor is still in the making, the task was seen as a bit complex. Working with an unfinished product may take the tester’s attention to technical challenges at the expense of content and fluency of the simulation.
To smoothen the path, the developers’ representative, who also happens to be the actor representing the patient in the videos, and the nursing teachers, who have written the simulation case, took the chance to meet face-to-face.
The goal is to make the 360ViSiEditor user interface so simple, that any teacher with just very basic IT-skills and some 360° videos, would be able to use it to create a 360-simulation. To develop the 360ViSiEditor further, a change of perspective is needed.
They will go through the simulation, create quizzes and check that the linking between videos tasks is logical. Next, the nursing teachers get to step into a student’s shoes and think of the simulation experience from their point of view. Progress needs feedback and ideas, and that is what the teachers are skilled at.
The 360ViSi project team was invited to the EU-CONEXUS PhD summer school in Zadar, Croatia, where the University of Zadar provided special training for PhD students about Professional and Scientific Communication and Networking in Multidisciplinary Environment.
The summer school had an interdisciplinary approach to research with practical examples for mind-mapping. The programme consisted of seminars and workshops held by experts, project proposals presentations, lessons on defining multidisciplinary projects proposals to fit the EU policies and the call rules and training about scientific communication and tools, resources and tips.
360ViSi: an international and interdisciplinary example
One of the invited lecturers was Dr Esther Navarro from The Catholic University of Valencia, who presented the 360ViSi project as an example of interdisciplinary research and teamwork.
Navarro portrayed the project as an example of how a team consisting of lecturers from different European universities, technicians and companies work together in the field of new technologies.
The project team works on several tasks. First, it provides teaching and learning tools to higher education using new methodologies. The second task is to develop solutions and content for e-learning, which is carried out by the universities’ and companies’ technicians. The third focus point is to provide new business opportunities for companies that elaborate services, products and technologies related to learning.
The 360ViSi team was introduced by presenting the different universities, tech groups and companies, the existing relationship among some of the members, the needs that originated our project, the type of project that it was and what it was about, needs and competencies within the partnership, interdisciplinarity within the team, barriers for interdisciplinary collaboration and adopted solutions.
An 83-year-old mother and a best friend, who is also a retired community nurse, agreed to take part.
Before filming, it was important to ensure the home environment reflected the details in the script, all the necessary equipment was in place to meet the clinical scenario requirements and that the ‘actors’ were comfortable with the scenario expectations. In the 83-year-old’s case, the necessity to appear dishevelled was possibly the biggest challenge of all!
Learning points from filming ‘on location’
1. Have a very detailed specification for filming each of the scenes including the following elements:
Positioning of actors and equipment in the scene
Lighting in the room
Where the person filming will be positioned
Ensure the actors are aware of the sequencing of each scene and give them a script to prompt what they need to do/say is really helpful
Equipment list including technical e.g., camera, stand, microphones, lighting, batteries
2. Visit the filming location beforehand to check out the space, potential issues (as far as possible – some can’t be predicted!). Also, if you are filming outdoors, it is important to check weather reports well in advance of any date decided for the shoot. As filming 360 video outside in the rain is not advisable.
3. Prepare your equipment in advance and make sure you have spare memory cards, batteries etc, and check everything is working beforehand. It is useful to create a checklist to make sure you don’t forget anything before travelling to your location. Get to the location in plenty of time to set up.
4. For our shoot we used the following equipment:
Insta 360 One R camera
iPhone 11with Insta 360 app installed (app available for IOS and Android phones)
Bushman panoramic monopod V2 – weighted tripod stand
Insta 360 selfie-stick & tripod
2 lavalier lapel mics & 2 Sennheiser mobile body packs and receivers
2 x 32gb SanDisk Extreme PRO SDXC (For Zoom – audio)
2 packets of replacement AA & AAA batteries
1 x Heaphones
The kit above is very portable and mostly fits (complete with travel cases) into a medium sized backpack.
Sound: We could have relied on the built-in microphone within the 360 camera for our audio and further reduced the required kit, but we already owned zoom and lapel mics and wanted to try and get the best sound possible. (Remember when using external audio equipment to provide an identifiable noise such as a handclap at the start of each clip once all devices are set to record. This helps when aligning/syncing the external audio with the video clip during editing).
Lighting: We decided not to take any lighting and rely on house lighting and portable table and floor lamps for indoor scenes. This worked well and looked natural and as part of the furniture when viewed in shot.
Remote control: We used the iPhone to take still photos of equipment and control the camera remotely but could have downloaded the Insta 360 app onto almost any type of smartphone.
Tripod: We did invest in the weighted tripod, this is advisable as other tripods we experimented with offered little resistance to even the slightest amount of breeze and could easily fall over, which could seriously damage the camera and at very least ruin the shot. The weighted tripod has also been designed to work well with 360 video cameras and leaves very little stand footprint to remove during editing.
Monopod: We also used the monopod for a motion shot with the nurse holding the camera at arm’s length whilst entering the house. Shots such as this can cause motion sickness when viewed in headset, but this one seems to be working.
Frame rate: It was decided that we would use the higher quality HDR mode setting for stills and all scenes were shot in 5.7k. at 30fps.
5. Remember with 360 you are shooting blind – be prepared for re-takes in an uncontrolled environment. We had family members walking downstairs into the shot, a cat walking through the front door! Tell your participants that this may happen when they agree to take part in the filming. If they have not learned a script, they have to keep remembering what they said or should be saying in the scene.
6. Remember when you are filming 360 the field of view is much wider than in usual filming – things or people you think are out of shot are sometimes in. We took advantage of natural hiding places such as the hedge in the front garden and the alleyway next to the garage to remain safely out of shot for the outside scenes. We monitored each scene via the Insta 360 phone app which was installed on the Iphone.
7. Take lots of batteries. A battery-operated kit is great to get the flexibility in how the scenes are filmed and for ease of use by the operator, but it is also useful to have an assistant to check battery levels in the different pieces of kit.
8. We had one professional technologist filming the scenes – with two assistants helping to set up the scenes, look after the actors/actresses and check batteries. It is important to check that the actors or any important information that you wish to include in your video is not captured in the camera’s stich-line. Having more than one person checking that each scene is correctly set-up is extremely useful.
9. Our participants in the film were an older family member (as the patient) and a district nurse (playing the district nurse) – they were not trained actors but were confident at role play. They read the specification of the scenes and activities beforehand but did not memorise a script but ad-libbed the scenes. This worked well in this case, but it is important to select performers who are confident to do this.
The new tools were recently demonstrated in an online workshop. See video from the workshop at the bottom of the page for a detailed explanation of the tools.
The purpose of the 360ViSi Editor, created by the Finnish 360Visi project partners Turku University of Applied Sciences, is to create a simulation by connecting short 360° videos. This is done by compiling nodes, the basic building blocks, together with actions that trigger the next activity based on the players choice.
The beauty of the tool is that it gives the user the possibility to create whatever construction the user needs or wants. Some editing is possible in the editor, for example, clipping and looping videos. What adds interest to it, are the features such as providing the player with additional information in the form of pictures of texts, a randomizer that chooses the next step on behalf of the player and questionnaires enabling to test the players’ learning.
The tool simply takes in the JSON files produced in the 360°ViSi Editor and publish them on the internet.
With the 360°ViSi Player, created by another 360ViSi project partner, a Finnish companyADE Ltd, the simulation creator proceeds to the part of publishing the content.
The tool takes in the JSON files produced in the 360°ViSi Editor and gives the user an interface to play through the simulation.
360°ViSi Player works on a browser, enabling to publish simulations on the internet and use them with multiple different devices ranging from mobile phones to VR headsets. The idea with this tool too, is to offer a simple and user-friendly interface. The user, usually a teacher, gets a link that can be shared with the players.
Want the detailed explanation? See the video of the demo here:
Imagine standing in an operation theatre and handing instruments to a surgeon. To nursing students, the impression can be nerve-wracking and may cause them to choose another area of specialization.
To guarantee that the future is not lacking nurses with competence to work in this demanding field, Turku University of Applied Sciences has created a virtual 360° game that supports the students’ skills in identifying surgical instruments.
The game gives the students a boost in their competence and showcases the reality of working in an operating theatre.
“When turning my head, I saw the very realistic environment. I believe this game lowers the threshold to work in an operating theatre,” says Jasmine Pitkänen.
Pitkänen, a soon-to-be-graduate nursing student. Along with her classmates, she gave an estimation of the learning experience. With a bit more practice on the game, Jasmine would be ready to jump on the deep end.
Learning by doing was a rewarding method for the students. Instead of reading books or watching videos and learning passively, the students appreciated active practicing and experiencing the lifelike situation. Or like Mikko Kinnunen put it:
“The experience was concrete, authentic and very realistic. I improved my score after rehearsing just once. When you practice, you learn.”
The students appreciated that, unlike books and videos, the game indicated incorrect answers. This feature along with the possibility to retry were seen as big bonuses from the learning point of view.
“If I handed a wrong instrument, it was made clear, and I got a chance to try again,” said Riikka Mörsky.
Instead of having the instruments explained as a list in a book, the game showed all the instruments at one glance. When reading about the topic, it did not occur to her, how the situation would look like in a real setting.
“If I had played this game at the beginning of my studies, I might have chosen to practice in an operating theatre.”
Valtteri Hannila appreciated the safe learning experience. Despite handing a wrong instrument several times, the surgeon stayed calm, and the rehearsal continued. In the real-life, a surgeon might not be as understanding. Playing in front of others added, however, more stress to the situation.
During their nursing studies, the students had played a learning game before, but this was the first VR learning game experience. Based on their practicing on the Instrument game, both the students’ experiences and attitudes towards the learning method were positive.
See video and interviews of the students testing the instrument game.
That led to a demo with the Norwegian University of Stavanger and based on their recommendation, the Finnish project team from Turku University of Applied Sciences was keen to hear more about the company and its products.
ONsim offers its customers VR headsets and easy-to-use 360° training solutions mainly to the medical sector whereas Turku UAS is, as a part of the 360ViSi project, both creating new methods for nursing education and developing the 360ViSI Editor enabling practically anyone to create simulations from 360° videos.
With a mutual passion for VR and simulation, the meeting between ONsim and Turku UAS was a sparkling start to the cooperation. Both ONsim and Turku UAS teams are enthusiastic about developing not only their products but also each other’s ideas. With cross-fertilization in mind, both the project and the company are looking forward to future cooperation.
With help from University of Stavanger, the Norwegian Police University College has produced and started using 360° video in education with the purpose of boosting asset recovery behavior from police officers, both bachelor students and post graduate police officers.
Inger A. E. Coll, assistant professor at the Norwegian Police University College, explained that police students need to be able to identify valuable assets during search and seizure of properties when a crime is suspected.
Learning about asset recovery
The 360° video simulation allows the students to move around in an apartment in search of valuable items. Before starting the tour, the students are provided with this context for the scenario: The couple living in the apartment are suspected of massive drug sales. They have no recorded income and receive social security benefits.
Placed in the apartment are expensive female accessories, jewelry, valuable art and wine, user equipment for cocaine and documents proving ownership of a horse and a yacht. It is up to the student to identify and assess them.
“The simulation is offered two times for the students. The first time, the goal is to boost their attention and stimulate reflection. Later in the course they do the same simulation but are then given added information about the items’ value, to increase the learning outcome,” Coll explains.
The simulation is not a compulsory part of the education, but since it was introduced, 90 % of the students have completed it, and their feedback is very positive.
360° live streaming
The second part of the workshop was called “Possibilities with 360 live streaming” and presented by Kåre Spanne, media engineer at the University of Stavanger.
Spanne talked about how 5G, the fifth-generation technology standard for broadband cellular networks, is enabling a new aspect for 360° video: live streaming.
In China, an intensive care unit (ICU) has started using live streaming of 360° video to enable family members to “visit” patients on the ward. See the news report about the story by CGTN:
“I believe we will see live streaming of 360° video used in education in the near future. One opportunity is to use it to observe students while they perform specific procedures,” Spanne says.
More workshops to come
This particular workshop was hosted by University of Stavanger, and all the partners will take turns to host workshops where a larger audience is invited to take part.
“The Knowledge Alliance is all about sharing knowledge. The project partners have insight into other cases that could be relevant and inspirational to the 360ViSi project, so this is a perfect arena for us to learn and develop our project,” says Atle Løkken, project manager for 360ViSi.
Ingrid Tjoflåt and Bodil Bø (UiS) together with Dr. Jane Rogathi, Dean of the Faculty of Nursing at Kilimanjaro Christian Medical University College in Tanzania.
– The project’s overall aim is to implement simulation-based education to strengthen the capacity and quality of nursing and midwifery education in Malawi and Tanzania, says project manager Ingrid Tjoflåt.
She is a professor in nursing at the Faculty of Health Sciences and will be coordinating the project. Tjoflåt has extensive experience from research on quality and competence development in various international contexts and research on teaching methods focusing on digital tools.
For the past three years, Tjoflåt and Bø have been conducting a research project on simulation-based nursing education with partners in Tanzania and Madagascar. Together with an international project group, they are also involved in the 360ViSi project.
–We will use experiences from this and other relevant projects in our collaboration with Malawi and Tanzania, says Tjoflåt.
According to the team at Screen Story, this was a great opportunity to apply interactive 360° video to a real user case. With the help of 360° technology, they were able to make the exhibition into a digital version – open to anyone, for free.
Explore the exhibition
The exhibition is still available online, so feel free to visit it. Navigate around and have a look at this example of one of the many ways to use 360° video.
In contrast to a normal video, the interactive 360° video allows you to choose what you want to look at and where you want to go, just as you would if you attended the exhibition physically.
The document provides basic information on producing 360° video using consumer-oriented equipment. It will discuss what is important to know when shooting 360° video, like tips about equipment, camera settings and camera position.
You will also learn about what you can do post-production; such as editing and exporting the finished video. The user guide gives recommendations and descriptions of different kinds of hardware and software that can be used, with links to several video tutorials.
After reading this – you should have the information you need to get started. We wish you the best of luck!
The European Commission’s support for the production of this website does not constitute an endorsement of the contents, which reflect the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.