playMaker

Author Topic: Leap: Who's playing around?  (Read 5106 times)

jermerqua

  • Playmaker Newbie
  • *
  • Posts: 22
    • View Profile
Leap: Who's playing around?
« on: April 30, 2013, 12:04:24 PM »
So i see some folks are playing around with Leap now and plugging into Unity

I see so much potential here. Especially for those of us who love Playmaker to do the heavy lifting programmatically. Does anyone have hands on a Leap Dev kit and/or more importantly begun to play around with some custom actions that more easily plug-in Leap as an input. Would love you to share via this thread if so. 

Lane

  • Administrator
  • Hero Member
  • *****
  • Posts: 2488
  • Yup.
    • View Profile
    • Cleverous
Re: Leap: Who's playing around?
« Reply #1 on: April 30, 2013, 12:31:37 PM »
Want... Most definitely want!

Is the sensor thing available already? I'll test the crap out of it if its a reasonable price for the tool.

jermerqua

  • Playmaker Newbie
  • *
  • Posts: 22
    • View Profile
Re: Leap: Who's playing around?
« Reply #2 on: April 30, 2013, 02:01:07 PM »
The device itself cost $79.99 I'd say that's reasonable.
https://www.leapmotion.com/
you can preorder but they will not be available until July now. Was Mid may but they pushed it out a bit. They are releasing the SDK soon to those who signed up for Dev program but are not releasing anymore dev kits currently.

Anyone out there tinkering with the dev kit yet? Anyone plugging into Playmaker. We should be all over this one.

Lane

  • Administrator
  • Hero Member
  • *****
  • Posts: 2488
  • Yup.
    • View Profile
    • Cleverous
Re: Leap: Who's playing around?
« Reply #3 on: April 30, 2013, 03:43:45 PM »
Didn't find anything on the devkit / sdk but I preordered the device. We'll see whats in the box.

jeanfabre

  • Administrator
  • Hero Member
  • *****
  • Posts: 14701
  • Official Playmaker Support
    • View Profile
Re: Leap: Who's playing around?
« Reply #4 on: May 02, 2013, 12:15:22 PM »
Hi,

They sent me a beta device for free! :) so playing around with it, and the problem is the same as with the Kinect, too much freedom and raw power to encapsulate into a few custom actions...

 the basics events you get from these device it never enough for anything above hello world type of projects, so I am very much questionning myself how to provide a useful library of custom actions that are useful for a large audience... difficult, so any starting point and ideas welcomes, I am all up for it.


bye,

 Jean

Lane

  • Administrator
  • Hero Member
  • *****
  • Posts: 2488
  • Yup.
    • View Profile
    • Cleverous
Re: Leap: Who's playing around?
« Reply #5 on: May 02, 2013, 12:56:13 PM »
Very cool!

I was also worried about that. From my uninformed perspective just assuming how it works... I would think that there needs to be some sort of shape recognition that identifies and tags objects in the space as controllers, then actions are waiting for input from the controllers and treating them like touch device joysticks. Input would be 3d, so maybe a normalized area for the local controller coordinates is fed in and stored so the user can manipulate it... Depends on what kind of data you get from the device I guess.

It would definitely be necessary to detect how far the controller has moved from its origin, maybe there could be some trigger actions for when a controller has moved to a certain area of the space - too far forward, backward, etc.. Kind of distance zones from the origin.

Actions to use those controller ID's vector information for inputs.

It may really only be necessary to convert the info you get from the device into data usable for the existing actions... Vectors and velocity will play a large part and they're already accessible.

jeanfabre

  • Administrator
  • Hero Member
  • *****
  • Posts: 14701
  • Official Playmaker Support
    • View Profile
Re: Leap: Who's playing around?
« Reply #6 on: May 06, 2013, 11:02:03 PM »
Hi,

 Thanks for your examples. The problem here is that providing such level of support means writing a proper framework, not just actions to covet the leap api, cause hands reacts much like touch events, it disappear and changes id as you move around your hands, so it's very difficult to encapsulate this into a set of actions.

 I guess I should start by just covering what leap exposes in the api...

bye,

 Jean

Lane

  • Administrator
  • Hero Member
  • *****
  • Posts: 2488
  • Yup.
    • View Profile
    • Cleverous
Re: Leap: Who's playing around?
« Reply #7 on: May 07, 2013, 04:18:59 AM »
Makes sense.

It may be that we have to wait until there is some extensions on the Asset Store that we can access.

Lane

  • Administrator
  • Hero Member
  • *****
  • Posts: 2488
  • Yup.
    • View Profile
    • Cleverous
Re: Leap: Who's playing around?
« Reply #8 on: May 20, 2013, 07:37:16 AM »
Hey Jean, just curious if you had time to get into the LEAP stuff yet and might have an update or if its something we can even put into actions.

jeanfabre

  • Administrator
  • Hero Member
  • *****
  • Posts: 14701
  • Official Playmaker Support
    • View Profile
Re: Leap: Who's playing around?
« Reply #9 on: May 20, 2013, 09:59:57 AM »
 :'(

man... I have a Kinect and a Leap that only take dust at the moment... Litterally no spare time for this at all. I am currently setting up my new office, so I will have a better place to have them always plugged without getting in the way, I might bet more tempted to play around.

 Having said that, it may also be because Kinect and Leap are still very much toys, The Leap device I have is UTTERLY COOL but it's also very limited in it's reliability, it's shaking, noise is obvious, and hands are lost very very quickly and the features that are rock solid only have limited usage really, as well as the really difficult task of educating the user ( both Kinect and Leap are facing this problem big time).

So, I guess, the perfect way to have this moving further is to stumble upon a proper budget and a clear short term goal that make sense for Kinect and Leap to be used and explored.

 Porting the api is no brainer tho, only a matter of sitting down with coffee and take each property and method and make it as custom actions... so that part if feasible, if time is available...

Bye,

 Jean

Lane

  • Administrator
  • Hero Member
  • *****
  • Posts: 2488
  • Yup.
    • View Profile
    • Cleverous
Re: Leap: Who's playing around?
« Reply #10 on: May 20, 2013, 10:16:00 AM »
Thanks for the update, I had heard from web reviews and such that the reliability wasn't up to par yet but it is still in beta. They've decided to keep the SDK sort of private it seems, you have to register as a developer and be approved to access the dev section of the website to get at the goodies but they aren't approving many accounts lately.

They are eventually going to make it all publicly available, I suppose we can wait until then and hope the beta yields some good improvements to the product.

Red

  • Hero Member
  • *****
  • Posts: 563
    • View Profile
Re: Leap: Who's playing around?
« Reply #11 on: May 20, 2013, 12:55:10 PM »
Well, if i may jump in here given that i've also had my eye on the leap motion interface device, i've been brainstorming ways to implement it... though, this is all just ideas though because i'm not a programmer.

so, i can imagine that if there were a way for the SDK to set up where to detect the tips of the fingers/stylus/chopstick you want to use, maybe having that translate into a raw vector 3 variable? with that, you could apply it to any game that requires interaction in the field (such as a RTS, third-person diablo-style click and even just a physics puzzler.)

from what i can tell, it seems that it's strengths are mostly about getting data in to the computer and that data can be then used for a lot of things... i think, however, given the nature of Playmaker, being able to set it up so that it can just extract the data in a format that playmaker could read and understand would be the best option...

some potential ways i can see it being used... if the SDK is doing most of the heavy lifting, having a way to detect when a hand is in there... then detect the joints on that hand and the tips of the fingers... then you can create a "hand" mesh that will align itself to those points so essentially you have a basic telemetry device. (mostly i can see this being used much like the mouse-cursor from Black-and-White... that "god sim" game that took the "god" part literally and became, essentially, one giant tamagotchi like game.)

though, not knowing how to code and not having access to the SDK i'm not sure how this would be done... so, in that respect, maybe to implement the Leap motion system, having a sort of "engine add-on" that will do that and just extract the appropriate information to the game in the vector details since it does look like it's able to get the raw vector information.

as for how to recognize gestures of the hands in game with Playmaker... well, all i can really think would be to have a "hand" system like explained above and the system detects the angles of the joints... and when an angle gets to a particular "gesture" (which you'd probably have to define manually or tailor a rig to understand it) it'd then toggle an event or conditional statement that you can use... so, when the angles of the joints in the "hand" seem to be approaching a sort of "pinch" gesture, you could then translate that into a sort of "grab" system where it would "pick up" a unit on the field (aligning it to the tip of one of the "fingers" with the appropriate offset) and when that "pinch" is released, it "drops" the unit (this is with an RTS perspective in mind... though i'm sure it could be applied to sim games like simcity as well.)

is the main issue with integrating it more about what to make actions to help detect it? if so, i'd probably want to have a bit of a tete-a-tete with someone that's much better at programming to help translate it so i could understand it more thoroughly.

... this and the Occulus are the two things that i'm totally jazzed for! the Leap for the fact that this could change the entire way we think about interacting with computers and the Occulus because it is a VR system that is priced so that it's well within the reach of the average consumer. (people spend around $400+ for televisions and monitors... hell, some are more than happing dropping that much on game consoles and significantly more on computer hardware... so, it's priced appropriately i'd think.)

jeanfabre

  • Administrator
  • Hero Member
  • *****
  • Posts: 14701
  • Official Playmaker Support
    • View Profile
Re: Leap: Who's playing around?
« Reply #12 on: May 20, 2013, 11:27:47 PM »
hi,

 the issue I am having in diving into porting Leap or Kinect, is the following:

 when you port actions to deal with touch events, or mouse events or joystick events, you have reliable data stream. With Leap or Kinect, it's unreliable: hands appears, dissapears as you move. So it's not a question of porting the api ( which has already all the things you mentionned really and is fairly easy, only time consuming), it's a following question:

Who is responsible to sanitizing the data, Leap, the framework built on top of Leap or the developer in the project itself?

Currently, with Leap and Kinect, this is the job of the developer, and I can't see PlayMaker regular user base deal with this. If a PlayMaker developer is expert enough to implement a system that will moderate data coming from Leap or Kinect, then creating the custom actions to work within PlayMaker is the least of her/his problem...

Do you see what I mean? I don't think that by providing a straight port of custom action to bind Leap or Kinect Api will be helpful to the majority of PlayMaker developers. It will be good to toy around for sure, but I don't think it can go any further than that, else requests will flood the forum. That would be a very expensive toy to play with and maintain.

 it's very much like setting up a character controller. Unity came up with several solutions, because obviously, there is a learning curve and some important expertize to acquire before building from scratch a character behavior. Leap and Kinect do not provide this, and I am not sure what is this next layer to build to make it viable and useful as the character controller package from Unity is. Yet the deal is far from answering everyone's need, cause most questions are still around setting up character to fit the need of a particular project or need...

bye,

 Jean

justifun

  • 1.2 Beta
  • Full Member
  • *
  • Posts: 228
    • View Profile
Re: Leap: Who's playing around?
« Reply #13 on: July 23, 2013, 01:23:06 PM »
That's disappointing to hear :( 

So there's no luck being able to use Leap Motion with Playmaker?


Red

  • Hero Member
  • *****
  • Posts: 563
    • View Profile
Re: Leap: Who's playing around?
« Reply #14 on: July 25, 2013, 01:41:25 PM »
I wouldn't say no luck... just that it might take time and a couple more people on the team at Hutong to crack this little thing.

So, Patience! besides, if the Leap does change everything then chances are it's going to gain much more presence in the consumer culture and when that happens it would be kinda suicidal from an economic perspective to neglect to at least try... but that's a big IF and also a long-game viewpoint.