tl;dr There’s a new content type for H5P called AR Scavenger. It features Augmented Reality. You can download it here.
Quite some time ago, Nele Hirsch approached me, asking if it was possible to create a content type in H5P that would allow to feature Augmented Reality (AR). From scratch? Quite a job! But in the wonderful world of open source software, there’s a framework called AFrame that allows you to use 3D models in a browser. And then there’s another framework called AR.js that builds upon AFrame and allows to blend 3D models into a device’s camera image. Nele pointed me towards that one. And now there’s AR Scavenger that builds upon AR.js and blends it all with H5P. It’s fair to mention that the soultion would probably not have been finish done now – but later – if the Hamburg Open Online University (HOOU) and the HAW Hamburg in particular had not partially funded this initial version.
https://youtu.be/yB36QZUh-wY
What can I do with this?
Just take a look! Here’s the content type that you can use right away, but you’d have to use it on a device that has a camera attached to it. Then just find the markers down below with your camera – either on another screen or print them on paper.
Marker 1
This one should simply show you a True-False-Question, but on paper you could put any H5P content type here.
Marker 2
This one should show you a 3D model of a duck that’s facing out of the marker, so if you have the marker on a screen, the duck should look right at you.
What is it good for?
So, obviously you can define markers and either let H5P interactions pop up whenever a marker is found, or you can blend a 3D model with the camera image. Not much, but using that features
- you could create some scavenger hunt game in a physical environment that requires students to find a marker to open a quiz related to the place of the marker,
- you could add markers to exhibits in a museum to deliver additional information in an Interactive Video,
- you could add digital 3D models to a physical book,
- you could …
I bet you will come up with even better use cases …
There are more sophisticated apps with a wider variety of features, but you usually have to pay for them in some way, and they’re usually not open source software. So, use this content type to take your first steps with Augmented Reality – and possibly work on software improvements, too?
How can I get it?
Ultimately, you will be able to install H5P AR Scavenger just like any other H5P content type from the H5P Hub, but the H5P core team will have to find some time for reviewing and releasing it first. Until then, you could download the required libraries (H5P.ARScavenger-library.h5p.) and upload them via the library settings page of your H5P plugin (not the H5P Hub where you’d upload content):
- Download the libraries file (https://www.olivertacke.de/labs/wp-content/uploads/2020/08/H5P.ARScavenger-library.h5p)
- In your H5P plugin, go to the library settings page
- Drupal: Content -> H5P Libraries
- moodle: Site Administration -> Plugins -> Activity modules -> H5P Libraries
- WordPress: H5P Content -> Libraries
- Find the “Upload Libraries” section and upload the file that you just downloaded. You should get a message like “Added x new H5P libraries”.
Also, you can get all the source code on github, but it’s spread across a couple of repositories starting with H5P.ARScavenger (+ H5P.AFrame + H5P.AFrameAR + H5P.AFrameOrbitControls + H5PEditor.InputRow + H5PEditor.ARMarkerGenerator + H5PEditor.ThreeDModelLoader + H5PEditor.ImportantDescription).
But there’s at least one catch!
H5P does not allow to upload gltf and glb files
H5P currently doesn’t allow to include files with the extension gltf or glb that are required for the 3D models. The H5P core team will allow this shortly. Until then, without some modification, you will only be able to use AR Scavenger for H5P content types.
On Drupal you can amend the list of allowed extensions in the H5P plugin settings, but not in other plugins. So, here’s the pull request to allow uploading gltf and glb files. The changes will point you to what you’d have to patch if you’re not running H5P on Drupal. If you want to do this manually:
- Know what you are doing
You cannot really break anything here, but if you’re new to this and make a mistake, you might get some sweat 😉 - Find h5p.classes.php
On WordPress, it should be located at your-wordpress-path/wp-content/plugins/h5p/h5p-php-library/h5p.classes.php. On moodle it should be located at your-moodle-path/mod/hvp/library/h5p.classes.php. If you’re using moodle 3.9 and above including moodle’s H5P integration, it should be located at your-moodle-path/h5p/h5p/lib/joubel/core/h5p.classes.php - Take a look at yourself and then make a change
Add gltf and glb as shown at https://github.com/h5p/h5p-php-library/pull/85/files
H5P cannot access the camera
This one shouldn’t be a problem in many cases, but it might become one. Browsers are becoming more and more strict when accessing browser features such as video or microphones and expect HTTP Feature Policies to be in place (also potentially causing trouble for Audio Recorder). There’s a discussion about this topic related to H5P at https://h5p.org/comment/35346. I am proposing a code change (for WordPress to be adapted to Drupal and moodle) that will allow to use browsers’ features within the H5P iframe (unless the server is even more strict and doesn’t allow them).
If your browser (on your server) simply won’t ask you for the camera, you may need this patch. A suggestion can be found at https://github.com/h5p/h5p-wordpress-plugin/pull/114. The changes will point you to what you’d have to patch on WordPress yourself. There’s no suggestion yet for Drupal or moodle, but if you can code a little, it won’t be difficult to temporarily apply the same approach manually.
Questions? Maybe I have answered them already
Sponsor Note
Die initiale Fassung dieses Vorhabens wurde gefördert durch die HOOU und die Behörde für Wissenschaft, Forschung und Gleichstellung der Freien und Hansestadt Hamburg.
The initial release of this project was funded by the HOOU and the Ministry of Science, Research and Equality of the Free and Hanseatic City of Hamburg.
OMG, that completely blew my mind!!
That is such an amazing feature, thanks a lot for developing it. I cant wait to use it somehow.
You can use it in so many different ways.
Glad you like it! What are the scenarios that you have in mind?
Hello Oliver,
I really would like to use your enhancement in Terms of AR for H5P. It seems amazing! Tried to install it like you described it on my Moodle site ( 3.10). I have got al lot of error reports. It says that I am not allowed to install the folders of your Plugin Library. Wrong content type. I am not quite sure whether I should allow other content type without knowing what will happen.
Would really like to use it!
Thank you for your help!
Hi Jan!
I am not sure how I am supposed to help here. You already tried to install the library knowing that it has not yet been reviewed by a third party, and you seemed to have been fine with that – what if moodle had not complained about a lack of rights to install custom libraries? What’s different now that you are obviously required to use some admin account?
Best,
Oliver
Hi Jan!
Me again. I just checked how the moodle core integration of H5P (which is not the same as the plugin) handles H5P libraries.
I assume you did NOT go to the library settings page as my manual suggests, but you went to a course of the content bank and tried to upload the file that I linked to there. That file is a library only, not some content, so moodle will not upload it there (which is absolutely correct).
If you instead download the demo content from the page and try to upload it, moodle will complain that there’s a gltf file inside that cannot be used (which is also absolutely correct as I described in the blog post).
If you are, however, on the library settings page as my blog posts suggests (you need to be an admin), then you can upload both an H5P content file (which is a bunch of libraries + some parameters) or the H5P library file without problems (which is how it is supposed to be).
Best,
Oliver
Hi Oliver,
thanks a lot for your help! You are definitly richt with your first answer to my request. I tried to install it in the wrong place at the site administration. I have managed to install your zip file now with no problems. It ist listed now as a installed H5P Type. I dont want to waste your time with moodle admin questions, but it didn’t show up when I trie to select it as a h5p-content type on the h5p hub. But I will ask our server Admin for detailled help.
Thanks Again and thanks for your contributing for the open educational ressource world and mor equality and chances in Education!
kind regards,
Jan
Hi Jan!
Don’t worry, I am not annoyed. It’s just that I am not an expert for moodle, let alone moodle’s integration of H5P that’s quite different to the H5P plugin for moodle. Unfortunately, the flexibility of sharing content (that may include new libraries that are not yet installed on the target platform) is limited by restrictive rights management – on the other hand I fully understand why administrators prefer it that way.
Wow. This opens up so many possibilities!
In my experiments, some glTF aren’t working so well.
https://github.com/KhronosGroup/glTF-Sample-Models
The avocado, for instance.
(Tried different formats.)
But then, some very complicated files do work.
https://h5ptest-aenkerli.profweb.ca/?p=25
Since I want to introduce this during a Zoom meeting, I need to figure out a way to make things work. Not sure I’ll share my phone’s screen, as it can get a bit complicated for something which is a small part of a longer session. Besides, I mostly want people to try it themselves.
I’ll probably share the QR code from WP and then display the marker. Was trying to think of an image that people might already have with them…
At any rate, I’m already thinking of a number of things to try with learners and learning pros.
Hi Alex!
The content type should be able to handle any model that an out of the box AFrame in version 1.0.4 can handle, as that’s used as a foundation. I have not included all kinds of loaders though as this was beyond the scope of the contract. I just tried the Avocado model (glTF 2.0 binary) and it works just fine. The only thing that I noticed is that you’ll have to zoom in quite a bit given the model’s default view parameters.
Hi Oliver,
thank you for that really valuable work! We consider the usage of ARScavenger for a “art bounty hunt” throughout a city.
Is there a (offiicial) way to enable interactive video as h5p content type using ar scavenger? As i saw, on the demo page at einstiegh5p.de this is possible there.
In my demo installation, normal video is available, but interactive video is not in the dropdown at the arscavenger content creation screen.
(interactive video as “plain” h5p content works)
where is the magic wand hidden?
Thank you,
Thomas
There’s no magic wand required. AR Scavenger is referencing version 1.22 of Interactive Video. If that’s not installed on your platform, then you cannot use it.
If you can wait a couple of weeks: The content type is currently being reviewed for release on the H5P Hub, so it will get an update soon anyway.
Hallo Herr Tacke,
Ich schreibe hier mal auf Deutsch, da ich des Englischen begrenzt mächtig bin. Zunächst vielen Dank des h5p-Typs AR Scavenger. Ich sehe darin enorme Potentiale für die digitale Lehre. Ich weiß auch, dass Sie kein Tech-Support sind. Dennoch will ich mein Problem hier darstellen: 1. Ich nutze für die Erstellung von h5p-Inhalten gerne https://www.lumi.education/de/
Hier ist der h5p Hub enthalten. Ich habe dann ihre Bibliothek integriert für Scavenger. Dann habe ich einen h5p-Inhalt mit AR Scavenger erstellt über den Hub. Ohne 3d-Modell um es zunächst zu testen. Dann habe ich das ganze als html-Seite exportiert. Dann habe ich die exportierte html-Seite normal mit einem Browser geöffnet. Die Seite öffnet, der Name wird dargestellt. Der Browser zeigt an, dass er auf die Kamera zugreift. Allerdings wird das Kamerabild nicht auf der Seite dargestellt. Ich habe auch probiert den Inhalt als SCORM-Inhalt zu exportieren und auf die Lernplattform meiner Schule zu integrieren. Auch hier dasselbe Problem. Nun weiß ich nicht mehr weiter. Ich kann im Übrigen den Testinhalt, den Sie auf dieser Seite erstellt haben, nicht mit Lumi laden. Danke für jede Hilfe
VG S.R.
Stimmt. Aber eine Konsequenz hat das Wissen offenbar nicht 😉
Das ist nicht verwunderlich, weil das gltf-Dateiformat (für die 3D-Modelle) in der veröffentlichten Version des H5P-Kerns noch nicht erlaubt ist und Lumi das entsprechend völlig korrekt die Datei als ungültig einstuft. Dass das Dateiformat in der jeweiligen H5P-Integration (hier Lumi) erst freigegeben werden muss, steht ja auch im Blogbeitrag.
Das tut mir leid. Das wird vermutlich daran liegen, dass einige benutzte Bibliotheken dynamisch gestartet werden (müssen) und der HTML-Export von Lumi das nicht umsetzen kann (und SCORM ist letztlich dasselbe in Grün). Das wäre dann aber erst einmal eine Frage an das Lumi-Team, nicht an mich.
Hey i just made my fist steps in moodle, and i wanted to try your really cool AR Scavanger. But i can not find it in the of h5p features.
I made also a trail account on h5p.com But even there i can not find it in the AR Scavanger when i want to create my own Content.
Am i missing something?
help pls
Hi David!
On moodle, I assume you’re using moodle’s own custom core integration. By default, it updates the list of content types every 30 days, so you may need to re-run the corresponding cron job or upload the demo content from H5P.org (as admin or user with permission to install libraries). Using the H5P plugin, you should be able to install AR Scavenger via the H5P Hub (if you’re an admin or a user with permission to install libraries). If you’re not an admin at all … ask your admin.
On H5P.com, I am not sure if AR Scavenger is active by default. You may have to activate it in the settings first: Manage Organization => Settings => Content Type Settings
Cheers,
Oliver
Hello Oliver,
Thanks a lot for your work! That’s really amazing.
I have a question: do you think it would be possible to use ar scavenger directly on a web page like this?
To show a 3d model without the need for user sign up or login
Thanks a lot
Best,
Melissa
Hi Melissa!
AR Scavenger will always require a trigger in order to display a 3D model, but one could create a simple 3D model viewer instead. The code is there essentially, but someone would need to put it into a new content type.
Cheers,
Oliver
Hello Oliver,
thank you for this great plugin. I have recently given a workshop on it at a moodle convention. The core functionality is great. There is one thing I’d like to ask you about. Models are generally displayed as black over the marker and will only show a glint of their texture or vertex color. I added light sources in blender and saved them into the gltf-file. I work in 3D so this is no problem for me. I wondered if it was possible on your end to add a default light to the AR scene, so that colleagues who are less proficient in 3D applications use downloaded models more easily?
Thanks for all the great h5p content you are putting out!
Cheers
Fabian
Hi Fabian!
I have never experienced any such issue, and if I recall correctly (did this years ago), it should not be required to set a light source at all, but AFrame would take care of this.
Sorry, but it’s very unlikely that I am going to work much on AR Scavenger. The main reason is that this was contract work done for a third party. That project is over: Offering a very basic yet functional content type that others can extend – I already did when building it, as the funding didn’t suffice. But I am not too invested into this content type personally, so I am probably going to spend my free time on other things.
Best,
Oliver
Hi,
I`d love to use AR scavanger for a interactive walk through a garden in a workshop for occupational therapists,
I tried setting it up on moodle and wordpress. Everything works. I like it 🙂 But the content types I can set up intercations with are very limited. In a video and several articles I have read and seen that there has to be a way to link more content types than the ones I get running. In my wordpress-setup I am only able to interact with “Table”, “Text”, “Image”, “Image slider”, “multiple choice” and “Single choice set”. Could you give me a hint how to extend this list? In particular I need to link to the types “interactive book” and “audio”. Is that possible? Thet would be amazing.
Thanks for developing this great content type. It`s great!
You can see supported subcontent incl. the respective version number listed at https://github.com/otacke/h5p-ar-scavenger/blob/master/semantics.json#L145-L177. If those don’t show up, you should ensure that they are installed in the appropriate version (which should happen automatically if you install content types via the H5P Hub).
Interactive Book is possibly not suitable for being used as subcontent (please cmp. https://snordian.de/2023/11/16/why-isnt-h5p-content-type-x-available-as-a-subcontent-option-in-content-type-y/). I wrote this content type as a contractor, but that content type is really not one that I like too much, so I don’t think that I will work on it for free any time soon. Sorry.
hi Oliver, thanks for your quick response. The types don’t appear, when I add them via the hub…well I am gone try and as I know that that’s the right way I will get it running for sure 🙂
Don’t want to convince, just explain:
I like the interactive book for it’s possibility to combine different types next to one another. let’s say a text with an audio description and an image slide underneath. other types “only” allow me to show one limited content which for teaching as I do it is sometimes a bit frustrating as it sets the boundaries a bit narrow. (course presentation would be possible for that purpose but is not as responsive on mobile devices…or simply setting a link to w WordPress page but that would force the user to leave the frame of at scavenger.maybe I miss another option here?).
I’ll find a way though 😉
thank you so far and have a nice Sunday
Henrik
I fully understand your use case. Interactive Book uses Column however, and that introduces content types that are not well suited for being used as subcontent in a content type like AR Scavenger (e.g. Timeline). Those need extra treatment in the code. I have tried to improve that situation by creating a so called pull request to H5P Group. That pull request would fix the situation for Timeline. It has not been lying there untreated for over a year now though. I already had to come up with extra fixes for H5P.Tabs which also uses Column. Coming up with workarounds for dependencies is not how it should be done, but the dependency should get fixed.
So, we have combination of me just being the contractor (not invested in the content type), me already providing fixes that would allow more things (ignored though) and me being expected to spend free time on things that I don’t really like (not too motivating). So, I’d not say “never ever”, but currently I’d not expect me to implement this.
thanks for your time anyways. AR scavenger is great the way it is. I understand the limitations (and your frustration about processes) within h5p…especially as you put time and effort in the topic.
For me it’s definitely pros over cons. I am loving the opportunitys that me as a teacher get with it and in most of the cases it fits my need.
so…thx again. 😉
Hi Oliver, as you seam to have an interest and motivation for everything working as it is supposed to I’ll share the struggles I experienced trying to get “audio” and “course presentation” running as content type with AR sc.:
Well first I recognized that some of the types listed in the code snipped you linked earlier appear in the drop-down menu when installed through the hub, others simply don’t (if you need a more precise list here, let me know). I tried installing the other types both before ar sc. and afterwords because I thought there might be a cache that’s not refreshed for some reasons or so: no effect. Next I tried different versions of the linked types and ar sc.: only difference is that in your first release there is one more content type in the list that I don’t know…something with Twitter in it’s name if I remember that right. I also tried moodle and WordPress in Firefox and Chrome: no difference on that. Versions are right (until the number after the first point) and outside ar sc. the named content types work just fine. Only thing not working is they don’t appear in the drop-down menu to choose the linked content type inside ar sc. and therefore can not be used.
Somewhen yesterday night I cloned your git repo and started looking through the file structure and at dime code snippet s…well what can I say:I can not do anything here on my own but compile it to a .h5p file again but what’s the point here. Just wanted to let you know I tried everything possible for me 😉
I would be thankful for any hints, where to continue as for me it would be great to get this working. I guess it just drives me crazy if I can’t get something working that potentially should be working…in this case getting this stuff running would improve the learning content I am planing a lot…so I can’t just give up 🙂
cheers
henrik
Loading dependencies is something that H5P core is responsible for, not a content type. The latter can just request them, but cannot offer them as subcontent if H5P core doesn’t load them. So, frankly, this is something for the H5P forums (where H5P Group listens). Since you’re the only one reporting this issue (and other sites run the content type just fine, e.g. https://einstiegh5p.de/), you should provide more insight into your platform configuration, how you installed the content type, etc.
I merely pointed to github to provide you with the list of content types that are supported. I didn’t expect you to build this yourself.
The Twitter content type was removed, as it doesn’t work anymore since an API change – and who wants to have anything to do with X nowadays anyway.
alright…I’ll head over to the folks of h5p to talk this through and find a solution. That nobody else reports that issue is part of what is driving me crazy about it 🙂 especially as the error occurs on both platforms (WordPress and moodle)..both are running on the same server/Webspace though…maybe I have to search the mistake somewhere there.
by the way: I didn’t want to use the Twitter-thing…just wanted to point out the changes I observed 😉 )
thanks for your help so far.