itch.io

I also forgot- feel free to join our discord (Neural Matrix Entertainment) if you'd like to contact me more easily. The link is : https://discord.gg/FxerWW9 . Hope to hear from you soon!

Hi Kendra!

I am not sure why it isn't showing on the submissions page for the jam...I'll double check through and see what's going on with it.

As for working with the BCI - Yes! We have gotten at least 4 kids to play through, and have their feedback recorded. We are working right now with sites in Edmonton and Toronto to get their feedback from the children who are judges at those sites as well, and I'll hopefully be able to get you some of the thoughts soon. (As well as let you know what is thought of in general!).

Anecdotally in the mean time, I can say that some of our kids enjoyed your game! In particular, they found the art and voice-over quite funny and liked that it seemed like a story. Talking with one of my colleagues in Toronto, she said she was quite partial to it as well, as she really enjoyed the 'storybook' aspect of the game. I think your instincts aiming for a younger audience was great, as it really does help it stand out as a unique experience. (Plus we all love the voice work :D ).

I'd be glad to work with you some more about how to make the game more accessible for the BCI. Right now, the biggest issue that is occurring is there was some miscommunication from my end on making selection options randomly flash . In particular, I wasn't very clear in our previous posts which has led to some confusion among various developers. There are two different BCI input methods being employed in the game jam - one which relies on random flashing, and one which does not. In your game, when the selection choices are flashing randomly it is difficult to time which selection you want due to delays in the BCI access method we are using for YummyYucky!

What could potentially help this (if you get time!) would be to switch to sequentially flashing choices (particularly in the 'Eating Food' section with the arrows). I'd be happy to discuss more with you too if you'd like to have a chat!

Hope you're doing well, and I look forward to hearing from you and updating you soon with some concrete feedback from the kids!

Cheers,

Eli

Hi H.H. Magoo!

We are currently putting the games together and trying to make them as usable as we can for the kids during the judging portion. I was wondering if you would be able to send me either the source code for YummyYucky or could send an updated version of the game where the choices are either presented sequentially or slowed down some (at least 2 second intervals)? Let me know if you have time, as we still probably are a month out from getting all the games finished! Thanks, and I hope your performance went well!

Cheers,

Eli

Perfect, thank you so much! Also that's a bit worrying the discord invite isn't valid...it is supposed to work continuously. Try this one-https://discord.gg/6T5wtBj

Hey team!

I have been trying to access your source code so I can work on this game some more, but cannot seem to find it. Can you message me on here, discord (https://discord.gg/n3Av7g) or send me an email so I can talk to you about how I can work some more on this? Thanks again for joining us in the jam!

Hey Jzraiti! I am trying to get your game up and running from the github repo, but it seems to be breaking... would you mind adding me as a seat on your Unity organization so that I can get access the game? If you haven't used Unity Collab before let me know and I can walk you through how to add me and/or share your project with me. My email is eli.kinneylang@ucalgary.ca.

Thanks and we are stoked to play your game! Glad you had fun:)

That's an interesting point- I think that would depend on what individual is focusing on, and whether your random animations/events would some how be overlapping with what is being chosen. 

So for an example- say you have your action selection icons (power/weapons/etc.), and those go from one color to another with high contrast. Great- that will elicit the P300 response. Now, if you have totally separate events occurring which float over those items it could be distracting and lower the accuracy of the P300 as they are now focusing on something which you aren't trying to have them target.

However, if your animations/events are happening around their peripheral that shouldn't affect the scenario- as that is exactly what the oddball paradigm does normally! Does that make sense? 

Stream is up and going! Check us out - Neural Matrix Entertainment on Youtube! 

Hi all- we are starting Workshop 2 with animation in Unity! Check us out - Neural Matrix Entertainment on Youtube!

Hey all! I will be monitoring this constantly throughout the weekend, so feel free to send me messages on here if you have trouble/questions/want some clarification.

I'll also be on our discord page hanging out to answer any questions you all have!

Hi Obsidias!

Glad to have you joining us from Minnesota:) I asked one of the teams in Calgary to reach out to you to try and potentially have you work together! Let me know if that works- if not, we can try and find some other groups on here or Discord to incorporate you in. Thanks again for joining us and I look forward to your submission!

Hey Anthony! Don't worry too much if you don't have a team yet. We will be promoting team formation tomorrow throughout the evening and will hopefully be able to help you find some people to work with:).

If you want, you could add some more information for others so they can know what your background is, what type of role you're hoping to help with in a team e.g. programming/art/music/direction/etc. (if you have a feeling for that!). 

Also, don't forget to join our discord page if you're working remotely, as we will hopefully be able to get some more people on there to chat in the looking for group sections!

Thanks for the question- the truth is I am not completely sure, but honestly think it should be able to (approximately 85% sure). On the back-end of things we are using a set of libraries and functions called Lab Streaming Layer (in particular super helpful codes from LSL4Unity) to handle the data transfer to/from Unity and MATLAB. The LSL data streams use a broadcast/subscription model so they can take any name you'd want. I do know we can have multiple LSL streams produced so it would just be a matter of subscribing to the right stream and having at least 2 instances of MATLAB open to process the 2 streams. Might be able to tinker with it throughout the jam to be sure. So in summary - haven't tried it yet but love the idea!

That's a great question! And, like a lot of the hardware parts of BCI, one that has a lot of variables with it.

For the single-button scenario we will be primarily using the commercial Emotiv Epoch system purchased a few years ago. This system is good in some design aspects in that it lets you control aspects of the 'click output' directly (such as hold time, and latency) but only to a degree. So I can change on the system itself how long it simulates holding the button for, (e.g. 20-500 ms), but it can only take a single value per keystroke. So if you have something like classic Mario where holding down the button will give you a bigger jump, we could only get one consistent jump size usually.

Now onto your question about latency - the short answer is the latency depends on how reliably the system is picking up brain signals. A good rule of thumb (from my perspective) is you don't want to design your game such that immediate twitch reactions (<1 second window) are required. Things like needing to have a sudden change with only one second to hit the button may be hard to accomplish, as the system has to 1.) Decide you are attempting to do a specific brain pattern as compared to the other constant monitoring; 2.) Decode what that brain pattern is; 3.) Send the keystroke with all the added information to the system.

This is all then complicated by the fact that the system is constantly monitoring so that sometimes spurious brain activities might be sent as 'intended action' when they weren't actually intended. In such cases the key might be pressed immediately with no delay, or might be sent multiple times in quick succession. Back to the Mario example, this is the equivalent of trying to do a running jump, but having Mario jump much to early only for you to be locked out of the 'right' jumping window. What might be more appropriate (from my naive perspective!) are things like QTEs with more generous windows (say 2-5 seconds), or 'bars' that fill up with multiple button presses.

Finally the 'next press' of the button again depends on how well the system picks up activity. If the person is continuously providing strong signal, we can actually select the latency between sending button presses so that we can increase/decrease the number of buttons sent in succession. (This is done again through the Emotiv's software). In our current set-ups this usually is only set at 20 ms, so if a person is able to generate the right signals repeatedly (and we aren't telling the computer to hold the button) multiple inputs will be sent in quick succession. 

Hope that helps some!

It has been my pleasure! I am always happy to discuss with others, especially with those who are so enthusiastic :).

I am sorry to hear you won't be able to participate in the jam (although you did provide an early draft of a game - so with your O.K. I'll let the kids play through it and try to get some feedback for you!). I would be happy to talk and work with you going forward in helping make BCI games. Feel free to join our discord at https://discord.gg/w3Zp3zN, as it might be an easier means for us to communicate. I wish you good luck in your performance and thanks again for your contribution!

Thanks for joining the jam! We are glad to have you:)

To answer your question- If you prefer to make a game with a single input rather than the P300 controller we have distributed that is 100% A.OK! We can use a different input method to have our children play your game without relying on the visual stimulus. So in that case we can definitely work with what you provide! (Right now we actually exploit this input method with an online knock-off version of Super Mario Brothers to play with some of our kids. We control the left and right movement, and they jump with the BCI controller).

This said, part of our draw with the P300 is in that there is a clear delay built in with the P300 that the user can see and expect. This same delay happens with the other input scheme (e.g. for single button inputs) but the delay/timing aspect is a bit less clear. On the plus side though we will be working with you (and other developers!) throughout the jam to try and help you make the best game you can and understand how these delays could be considered in different game design elements.

Hope that helped some! We are working hard to put together a Discord so that the participants who aren't in the Calgary/Edmonton area can chime in and ask us questions throughout the jam. Let me know if you have any more questions! Cheers.

Hi there- I wanted to dive a bit more into some of your questions. Feel free to keep asking more!

First off, I want to say I enjoy the concept of your game, and your enthusiasm for learning more about the BCI control schemes/input mechanism. Let me know if what I say doesn't make sense or needs more clarification.

Previously, when I mentioned using either the imagined thought pattern or the P300 as control schemes I wasn't sure what you were using for coding your game. The reason I had brought up the P300 is that the type of game you are proposing would work well with the P300 scheme. The problem, however, is that for the P300 to work you need to carefully time when stimulus occurs (e.g. the exact moment a highlighted option happens) so that you can look 300 milliseconds later in the recorded brain data to check if there was a response from the individual. (If so, then that was the option the child was focused on! If not, then they were probably paying attention to a different option). This timing aspect and pairing with the recorded brain signals is what we have aimed to take care of for game developers with our P300 tool in the Unity game engine. Since you are using Delphi (and I am completely ignorant to its code base/available systems etc.), I am not sure how to help implement a similar system from Delphi as my background isn't in networking or TCP/UDP at all. For that tool, we are using something called LabStreamingLayer which takes care of the network communication aspects for us. I briefly looked, but couldn't see anything from LSL which has been implemented in Delphi. (If you find one though let me know and we can try to figure it out!).

As such, you might be better off using your previous idea of sequentially highlighting the options, and we can then use the key-mapping to have a particular cognitive thought pattern map to the "Space" input. Doing so might make the selection times a bit slower and possibly more unreliable (depending on the time spent waiting on each given option), but it will definitely be able to work. With that in mind, I have tried to answer some more of your questions below, and hopefully provide some more clarity for you.

Burning Questions:

1) In my game, I present highlighted options in random order at a rate of 1 per second. Is that a good speed or would faster or slower be better?

The 1 option-per-second is fine, although you could speed up that flashing rate if you'd like (say 2 Hz - 8 Hz). There is a variety of literature out there discussing the different ways to 'optimize' the P300 response, but the "best" flash rate is variable across participants. Despite the variability, slower flash rates do tend to increase amplitude of the P300 response which means a stronger response which is easier to pick up. (Note- typical information on BCI responses are usually reported for adults, so depending the child's age and visual abilities slower flash rates can definitely be beneficial!).

2) In my game, the number of options available gets as high as twelve. In that case, the player must wait an average of 12 secs for the option they want to come around or come around again. What have you found to be the upper limit in the number of options / length of time between repetitions? User attention span must wane at some point. 

Regarding options-12 options is relatively small, and definitely manageable, in terms of P300-based applications. Many typical applications of the P300 in BCI is a spelling board. This often includes at least 28 choices (alphabet + space + backspace) arranged in a matrix. Some of our participants have been able to use such boards with moderate success, so 12 options should work fine. (Also having played through your game, it seems like it should be ok! Of course we will find out for sure when we do the jam and get some of our participants to provide some feedback!).

That said, you are right in terms of attention span being something worth considering in the development aspect as we are working with children at a variety of developmental levels. Hopefully that's what your game mechanics and design will help improve!

3) There is one circumstance in my game in which the player has only two choices (in the cardinal direction walking part if they are stuck in a corner). Can the binary randomness of just two choices generate a P300 response?

Yes- binary selection is a classic way to implement the P300. All the P300 response relies on is the brain's categorization of new, unexpected events which occur when you're paying attention to something (e.g. a picture/sound/etc.). So if you have 2 different options or 10 different options the P300 can still be elicited as your brain really only cares about the 'surprise' stimulus happening to the option you're focused on.

4) In my game, I tried three different systems for highlighting options. Choosing an animal, there is a yellow glow, a written word label, and a spoken word. Choosing foods, there is the same glow and a wiggling animation. Choosing a cardinal direction, the arrow changes colour from grey to red. Are these all equally effective? 4) In all three cases, the currently unhighlighted options remain on screen. Would it be better if only the current option was visible?

This is a pretty complex topic, with a lot to unpack. The short answer is that different methods for inducing the P300 event does affect its magnitude, and thus ultimately its ability to be identified in a BCI system. It's worth noting that the traditional method for inducing P300 from visual aspects involved high-contrast changes of squares around the letters in a spelling board (e.g. from black to white, similar to your arrow-scheme). But, that doesn't mean that the other ways you've chosen to highlight options isn't helpful or effective. For your purposes, I think its definitely worth trying to increase the contrast where possible to elicit strong responses (e.g. make the glows brighter/contrasting to the background). Additionally, the use of a written word label and spoken word is (in my opinion) very helpful in the animal selection task. To me, it helps keep attention and provides plenty of clarity/fun for the game.

5) I am a former kindergarten teacher and designed my game with a kindergarten-level audience in mind. Are there children in your group for whom that is a good fit?

I think this is a good fit! Please see my other post about the appropriateness of the audience. I think its good to have a range, and think there is not nearly enough consideration given to applications for very young possible BCI users. It is an important group to include in pediatric BCI research as it goes forward. (And one I am keen on exploring!).

6) There is some written language in my game but it is all conveyed verbally as well. Should jammers aim for a particular reading level?

This is another difficult (but great!) question- We have a wide range of developmental levels in the children who use BCI. Some are non-verbal, while some have full control over speech. As such, reading levels and literacy is quite varied for our children and something I will have to get back to you on after talking to some of the OTs and clinicians more about. That said, having the verbal option available for the game is quite nice as some of the BCI users do have impaired or obscured vision in certain areas, so having the choices reinforced verbally is helpful.

7) I program in Delphi so I don't use Unity and I was unable to try your P300 tool. Are there important  aspects I missed?

Please take a look at my other message to highlight some thoughts on the P300 in your game! My only other comment after playing it a couple of times is that spatial distance between options can affect the P300 (e.g. if options are too close/overlapping you might get false-positives). So you may want to spread out the animal selection options just slightly, or have them staggered in the screen!

Hope that helps, and you had a good weekend! Looking forward to your actual game submission!

Hey there! Thanks for all the questions- I'll follow up more thoroughly on Monday when I have some time.

I completely understand the desire for having a game that works 100%, and we are hoping to release our P300 tool in the next couple weeks so people can try it out and send us technical help questions (so you are ahead of the curve!). I may try to put together a 'releasable' example as well, so developers can get an idea of what might be a good way to structure the P300 (depending on my workload though!).

I briefly looked at the prototype, and just wanted to share some quick thoughts. First- I think it looks great! I can't speak for all the centers involved regarding what our audience's ages actually are (there will be 2 other hospitals besides the one I work at who we have partnered with who have BCI participants who will be our judges), but we definitely have a range of participants. As such, I am glad to have a game that is targeted towards the younger side of the population!

There are two main concepts which jumped out to me from your prototype which could benefit in using the P300. (Also working on some general videos to release so that these ideas can be explained a bit more from a different perspective!).

1. In any of the given selection periods, if you can have a 'count-down' cue prior to the selection time, it helps participants focus and know that a 'selection' period in the game will be coming. So just having some instructions saying "Get ready and look at the cross!" or something akin to that, followed by a count down before the selection time would help a lot.
2. Similar to 1- if you are randomly changing the choices, have a set number of 'times' that each choice flashes to break up the start and end sequence of the selection point. So for the characters, maybe highlight each character 10 times. Then have a break before the next 'selection' part begins. This helps on the engineering back-end as well, as we know when to start looking for the P300 response. Does that make sense?

I hope those help for the moment, and I'll answer all the other main questions more thoroughly soon! Glad to have your enthusiasm:).

As a side note (and a bit of extra information for you!), the choice/highlighting input scheme you are describing could be accomplished using a couple of different BCI input techniques. 

One, as you have mentioned, would be from the user causing a specific spike in brain activity (e.g. imagining a specific movement). Alternatively, the P300 tool we will be providing is well-suited for this type of game-input. When you highlight each of the options, if you do it in a random order multiple times you will actually elicit a well-defined brain response (the P300 response) which we can identify. Each time the object the child is focused on is highlighted the P300 brain response would occur and we could then select this option for the child. We are working on some videos which (hopefully) explain the P300 better! They will be added to the main page of the jam.

Either way, throughout the jam we will be available to help you with the BCI input design aspects!

Sorry for the confusing response- Yes, you are correct on multiple points! First, currently the Mario game we have cannot be played independently. We are currently helping our participant play by doing the running when the key-map is set for them to "jump" or vise versa. My example was to highlight that we can re-map to any given key for a game, e.g. changing the brain-input to be mapped to "forward" instead of "jump". 

Second, as you mentioned our kids cannot play any of these games independently. Typically the games have multiple input requirements with specific timing , which is much more difficult for the participant to handle, so we typically need to step in to help. That's one of the big inspirations of this game jam! Designing games with single inputs will let them play completely independently and help give them a greater sense of independence. 

Regarding your game proposal- that sounds perfect! I think our participants would enjoy that quite a bit.

Good question. The short answer is you can choose whichever keyboard input you prefer (space/wasd/arrows). Just make sure you document it at submission!

The longer answer is the hardware we will be using to evaluate the single-button input games allows us key-map specific thought patterns to any button on the keyboard (mostly). Since we have to train/set-up the hardware every time a game is run, it is simple enough to just use whichever input was coded by the developers. For instance, right now we have a Mario-like running game and have mapped onto "W" for jumping. If one of our participants wants to run instead (or say we use a different game), all I have to do is switch the input key (done from a GUI) or load in a different saved key-map.