Show HN: iOS app that corrects your form in real time using your phone's camera
firefly.fitnessMy friends and I made an app that uses your phone's camera to track and improve your form. It'll give you visual and verbal feedback for every rep in real time, ensuring accountability. https://apps.apple.com/us/app/firefly-fitness/id6464440707 (No, we don’t store or view any camera footage as you're working out.)
The challenges and programs are free to try, no sign-ups or subscription required. (We do ask that you sign-up to save your stats). Just swipe down on the paywall. As a heads up, challenges and programs have pre-set strictness, so you’ll need to perform them with proper form.
I created a story-based workout where you start in a room, progress through increasingly difficult levels, and earn badges. I wanted to make the workouts more engaging - Solo Leveling and Lord of the Rings were a hard inspiration! Let me know your thoughts!
Background
We're three friends from Virginia Tech. Starting our journey, we had no idea what to build...until our CTO, who had never worked out before, created a rep-counting app to help himself. That’s when we realized we also forget our rep counts! We looked at other fitness apps and felt they were all the same, just with different designs. You can check out our building process here: https://www.youtube.com/watch?v=j5CAK40wTwo&t=195s
But, after talking to a lot of people, we discovered three major issues:
1. Proper Form: It's hard to know if you have proper form until someone points it out. You either watch tons of videos, use a mirror/record yourself, or rely on a buddy to check if your form is correct.
2. Cost: For beginners who can't afford personal training services or smart gyms, this gives them an alternative option
3. Accountability: Our app only counts reps done with proper form using full range of motion (depending on the form strictness level you set).
It took a looonngg time...but we finally got the core feature working! We prioritized building a reliable pose tracker that accurately tracks fast movements. (Our old versions lagged in its detection, making the feedback triggers unreliable...cause you'd want to hear the feedback as soon as possible). Because of this, we were able to create our own exercise library, allowing us to control when and where to trigger feedback for form mistakes like 'Go Down lower' or 'Raise Your Right Arm Higher'. Also, form strictness adjuster and workout customization are a premium feature. But the challenges and programs are free.
Please exercise safely and would appreciate feedback! Hope to help a lot more people, please let us know what helped you start your fitness journey! Thank you!
I wrote software for the Amiga that did something like this for golf swings a few decades ago:
http://zoom.interoscitor.com/PetersonEnterprises/resume/logo...
http://zoom.interoscitor.com/PetersonEnterprises/Consulting/...
http://zoom.interoscitor.com/PetersonEnterprises/resume/down...
It could adjust the stick figure size to match the persons height and controlled a video tape deck via RS-232 to single step the video. The stick figure data was digitized from films of golf pro's at a high frame rate and the frames that best matched the 1/30 second frame rate of video were used. A genlock superimposed the graphics over the video. Wrote the software under contract and got a percentage, but the business never took off. It was somewhat cumbersome to use (roll the big rack shown in the picture out onto a golf course), they charged a lot, and they had no proof it actually helped improve golf swings, though they gathered some testimonials. The results of an analysis were difficult for a customer to use as well, for example "slow down your swing in the early 8 o'clock part of the swing", how do you do that? How much do you do it? It was difficult to translate the differences in swings into a kinesthetic sense of what your muscles should be doing. Perhaps adding an accelerometer wrist/ankle strap could provide additional data that would help with that.
Wooowwww! It's awesome to see vision tech being used like this! How long did this take you?? Also, is the club being used at all or just the person's stick figure? Oh and did adjusting the stick figure to match the person's height affect the feedback given? Like if someone was shorter, would it trigger cues like "Bring your club up higher on the backswing" compared to a taller person?
We talked with a baseball academy facing a similar issue where their previous software analyzed recorded videos really good, but the analytics were too confusing for coaches to understand and took too long to set up.
From what we learned, form is the main aspect. Regardless of speed, form varies by individual, and any deviations can impact the results. In our case, we fire a feedback like "Bring your hips down lower" after every rep attempt until the user completes their workout. From working with martial arts instructors and their students, short and direct feedback that guide you work the best!
And yes, would love to integrate more data for the user! Maybe using the user's apple watch features! Thank you for sharing!
No clearly labeled pricing link or mention of how much it costs. Stop decepting people like that and simply tell them what they obviously want to know, that is the best marketing you can do. Dark patterns are violence towards other human beings. Thank you.
Otherwise it looks great, but that turned me away. Yes, I know Apple forces you into transparency about it on the app store, but I don't buy products from companies that employ these patterns.
The app store lists the prices (I think it might be mandatory to have the prices there), but here you go:
- 1 Month $9.99
- 6 Month $39.99
- Ultra 3 Month $24.99
- 12 Month $19.99
Thank you for taking the time to post this!
I agree that it's extremely annoying to not have pricing up front and that hiding it is a dark pattern, but calling it "violence" is at best comical. Can we please reserve it as a term for instances of actual measurable harm if not physical harm?
[flagged]
Gotcha! Thank you for the feedback! The challenges and programs are free to try! You can swipe down on the paywall if you want to. Appreciate the honesty and will apply that, hopefully, some other time! :-)
Tracking your form in real-time is counterproductive. Kinesthetic awareness is essential to compound exercises like squats and deadlifts, and there's no way you are focused enough on the movement if you're doing reps while also watching a selfie. However, a post-set analysis could be really useful and be a great IAP option or reason to pay for a subscription.
Oh yeah, totally agree! I'm most likely not interpreting your comment correctly, so apologies about that!
People can workout on their own so tracking in real-time vs watching yourself through a mirror isn't much different. We made is so that the app audibly tells you how to correct your form in real-time (make sure to turn up the volume!)
So instead of uploading a video and wait to receive your analysis, you can correct yourself on the spot and make adjustments. Post-analysis is a great option for in-depth analytics but can be confusing to interpret. This was mentioned during our conversation with training academies. Again, great if you want more in-depth knowledge for sure!
Thank you for taking the time to comment!
chicken before the egg. its reasonable to want to use this to ensure proper form before trying to strength train with said form. Every pic on their website shows zero weight or bars being used and there is a focus on getting good form. even if i was pushing weights to near failure, id want to know the second my form start tanking to avoid injury. proprioception alone is not enough to ensure good form until you can develop the awareness to maintain it.
I'm gonna go one further: giving a shit about form is a CIA psyop designed to keep you from getting stupid dummy strong.
Loll yeah do whatever works the best for you! Think it's decent advice to start with general proper form and progress to other training techniques. Stay strong man!
Caring about form is what those soyboy vegans want you to do. Real lifters push through the pain caused by poor form. Let the wolf out.
Injuries while working out? That’s simply a skill issue
> No, we don’t store or view any camera footage as you're working out.
Does that mean the processing is local on the phone? This needs more information of what data leaves or doesn’t leave the device.
Yes! No .mp4 or .mov video is being sent to a server for processing back and forth. We do store your stats to display on your device. If you don't sign in however, the app doesn't save your progress.
How does this work, do I do the pose and squint or can I attach to a larger monitor? The example shows just the phone so I’m assuming I put it on the corner and then have to relay on a detector or color that says correct or not correct?
This is how I know I’m old. :(
Is this simply using a stock full body estimator (like mediapipe pose estimator[0]), or do you actually have custom ML systems trained on proper form?
I ask, since I don't think a general body pose estimator is fit, and an app potentially giving improper pose advice for exercise might do more worse than good.
If you have something substantial, please provide credible evidence, research and benchmarks.
[0] https://mediapipe-studio.webapps.google.com/home
Oh yeah! Thanks for asking cause this was the dilemma that we were experiencing at first. When starting out, think we were using mediapipe from ML Kit (Google) and CoreML. If you want a visual example of our first version, here it is: https://www.swivi.app/.
We started out on Android but switched to iOS because everyone in our area had iPhones lol. In short, the skeleton (the white lines) kept flickering a lot so we made our own implementation. Feel free to check out our iOS version history too!
You can see our tracking capabilities from this stupid short: https://www.youtube.com/shorts/cGJs9pJW_nE (Skip to the last part where I wave my arms around quickly)
You're very right, the general body pose estimators are laggy and doesn't track very well. We worked full-time to implement our own solution that took YEARS XD. So this isn't smth that we just made. Admittedly, we were focused on building which I know isn't the best approach. But..at least, we got this tho ahaha
For our full iterative building process, you can view it here: https://www.youtube.com/watch?v=j5CAK40wTwo&t=17s
But the most important aspect that you pointed out which I liked is safety. Like all AI, you shouldn't completely trust it 100%. The app is good at what it does, but it's just a tool that can assist your needs.
I tried to keep it brief, so I hope this is answered your points. Appreciate you!
Hi OP!
Great work, will try it out and get back to you!
I also wanted to build something similar but never implemented it fully!
The main problem I faced was, how do I get training data that represents correct form. Also when I started, I wanted to train it on compound lifts like deadlift, benchpress and others. But as someone else suggested, form is quite subjective and lot of videos vary a lot! How did you tackle this? Also for form checking, do you depend on things like knee ankle, slacking etc?
Appreciate it! Hope to hear the good and the bad!
Feel free to share your project and why you started! I just met someone who wanted to build a smart mirror that counts reps. Think the idea was sparked by using the Vision Pro during their workout.
As far as training data, use yourself! Free, you have control of the camera, and you don't need to scour the entire internet for the perfect and ideal form with the perfect camera angle - it's not like you can't move like them. If you can't...then find someone who can XD
There's not going to be the one, absolute right way of doing each and every exercise. The general advice is full range of motion (going all the way up and down). Now, when using yourself, it's good to have a third-party give objective feedback on your form. So, I wouldn't advise you to just film yourself and use that off the bat.
We include a form strictness level during the workout session (gear icon), so reps are only counted based on the strictness you set. This lets users control how precise the AI is when counting their reps.
Oh man....having the user choose from like 10 different push-ups wouldn't make too much sense (I'm assuming) since most users wouldn’t know the difference. Instead, we use a default push-up and let users adjust their strictness preferences for a more convenient experience. By default, I'm referencing to how the average push-up form (i.e.back straight, hands near your chest, legs straight, etc.).
Yeah! We look at the white dots on your body to provide feedback! Hope this answered your questions! Thank you!
Have you seen Kaia https://kaiahealth.com/ ? How does your tech compare?
That looks great, thank you. Just a quick Q in case you are aware: any apps/services that can help me formulate and do (daily) exercises about fixing a straightened lumbar lordosis?
Yes! Honestly, I don't know. I haven't seen a raw demo outside of the website itself. From initial looks, it looks a little bit jittering and...slidy? Idk if they're playing around with the EMA smoothing but I can't fairly assess it without more examples from them. Sorry, I can't give a fair answer to this question. But thank you for sharing!
This looks very promising! Being visually impaired, I often worry that I miss things about form when watching workout videos demonstrating an exercise. If this app could give me feedback to improve my form, that's wonderful.
Yes! Just make sure to turn up your volume so you can hear your feedback! Full honesty, we didn't think too much about the visually impaired. We do highlight the mistakes you made on your body using red. If there's something that we should be considerate of as well, would appreciate your input!
Challenges and programs are free try! Just swipe down on the paywall. As one commentor mentioned, there is a paywall for workout customization features. Thank you!
How do you determine what is "proper form"? All forms are arbitrary until a best one is picked, although that's highly variable to specific individuals. Did you work with licensed physical therapists to determine this? After working out for many years I'm highly skeptical of any app claiming to do this unless someone licensed is willing to say they back the results.
The only mention I can find is: "How do we determine proper form? Firefly references the average exercise performed by credible fitness experts."
Also like others mentioned, your pricing is incredibly deceptive. Make it front and center to users, otherwise "Full Access (7-day free trial). Cancel anytime." is a hard no-install for me.
Yes, you're right in that there isn't a textbook example of 'proper form'.
I appreciate you finding the quote on the website! For clarity, at the bottom of the homepage's FAQ section ("I already know how to do my exercises. Is this still for me?"), we’re not here to teach proper form since everyone is different. We just assist with accountability and using full range of motion. For premium users, we do have the option for users to freely control their strictness level when the app counts your reps.
The way we set the AI is by prioritizing certain body parts. For squats, we focus more on hip lowering than arm positioning. It's just making sure: Are you going all the way down?
No, we didn't work with a licensed physical therapist. We had certified trainers try it out and hear their feedback. This is just meant as a tool to keep you accountable on your form. If it's not to your liking, that's fine! Just trim down the level to your preference.
Last yes, we do place the pricing upfront after the onboarding pages. The challenges and programs are free to try after you swipe down on the paywall. I just didn't communicate the instructions properly. If you ever change your mind and want to give us constructive feedback, would love to apply that and help others!
I appreciate the comment and hope you have a good one!
>we’re not here to teach proper form
My friend, the title of this post suggests that the app can correct your form.
Do you use this app when you work out?
Yep! This is just to inform that everyone is different so we're not teaching the one right way. That's why we didn't set it at 100% form strictness for every one of the exercises. We just correct your form to the exercise shown, but no means claim that it's the one true way.
Yes! Not for every single workout since I've been working out for a while. My friends and I occasionally do squat challenges and pull up challenges for fun. Think we set them at like 100% strictness whenever we do them. I have one where I do 100 squats at 70% strictness since it takes me like 5 minutes to do them. Shreds my legs lol
can this app give an audible alert when form falls beyond threshold?
Yes, it will give a negative sounding 'beep' noise and provide you an audible feedback to improve like 'Bring your left leg higher'. Just make sure that your volume is turned on!