Home
Will swapping your lid make you faster?

Is it worth spending your hard-earned cash on an aero road helmet? We're going to check out everything that might influence your decision.

It’s all about the aggregation of marginal gains in cycling these days. You know the theory: make a small gain in everything you do and all those small gains could add up to become a winning margin.

Bontrager Ballista helmet - side

Bontrager Ballista helmet - side

But just how marginal is the gain you can get by swapping from a standard bike helmet to an aero road helmet?

The claims

Brands often make claims about the amount of time you’ll save by switching to their aero helmet.

Smith, for example, says that its Overtake helmet will save you about 25secs compared to a non-aero Giro Aeon over 40km (25 miles) at 40km/h (25mph) (Smith presumably means that a rider wearing the Giro Aeon would be going slightly slower than 40km/h at the same power output as an identical rider in an Overtake doing 40km/h).

Louis Garneau, on the other hand, released a paper when it launched its Course helmet in which it said that if a 70kg rider on a 9.1kg bike covered 40km in 53:20mins wearing a regular road helmet (stick with us!), the same rider would cover the distance in 51:22mins in an (unnamed) aero road helmet (presumably at the same power output), and in 50:40mins in the Louis Garneau Course.

Louis Garneau Course Helmet.jpg

Louis Garneau Course Helmet.jpg

In other words, according to Louis Garneau, swapping from a regular road helmet to the Course would save that rider 2:40mins in that scenario. That’s a huge chunk of time!

Specialized has what it calls its Win Tunnel – the company’s own wind tunnel at its California HQ. You might have seen its short videos that cover things like whether having a beard or shaved arms has an aero effect.

Specialized compared the performance of its Prevail standard road helmet, Evade aero road helmet (below) and S-Works TT helmet in the Win Tunnel and concluded that switching from the Prevail to the Evade would save you 40secs over 40km (25 miles).

Switching from the Evade to the S-Works TT helmet would save you another 20secs, but you’re not going to be using one of those for standard road riding/racing.

Specialized Evade.jpeg

Specialized Evade.jpeg

This testing was done on a time trial bike. The results would probably have been a little different on a road bike. In its sales literature, Specialized actually claims that the Evade would save you 46secs over 40km compared to a standard road helmet.

At what speed? Watch this video to find our why Specialized believes that the time will be more or less the same whatever the rider’s speed. 

Specialized also says that a pro putting out 1,000 watts in a 200m sprint while wearing the Evade will finish 2.6m ahead of a rider in a Prevail putting out the same power.

The cynical among you might think that Specialized has an interest in exaggerating the aero performance of the Evade, but the S-Works Prevail and the S-Works Evade cost exactly the same – £159.99 – so there’s probably no benefit in the brand promoting one over the other.

And in the real world

Our man Dave Atkinson took the Giant Rivet aero helmet to road.cc’s local cycling circuit in order to find out whether wearing it would add speed compared to using a standard road helmet.

Giant Rivet Bicycle Helmet.jpg

Giant Rivet Bicycle Helmet.jpg

Over to Dave:

“This is a graph of my laps of Odd Down circuit wearing the Giant Rivet and a standard helmet for comparison, a Scott ARX Plus. I was using a set of Garmin Vector 2 pedals to measure my power, and the power spread is from an easy 150W average for the 1.5km lap up to about 270W average, which for me is going pretty hard. For reference, that's a fastest lap of 2:22, which is just under 4th cat race pace (2:18 on Saturday, when conditions were similar).

Giant Rivet vs Scott ARX Plus

The graph shows power (watts) on the X (horizontal) axis and km/h on the Y (vertical) axis.

“What do the numbers say? Well, the trend line for the Rivet is above the trend line for the Scott, and depending on where you measure the offset you're looking at just under or just over 10W of difference. Put another way, 2-3 seconds a lap. 

“It's not a rigorously scientific test and there are always going to be other variables at play in an outdoor environment, but it does suggest there's a measurable difference in the real world. Every little helps, right?"

Ventilation

Many aero helmets in the past have lacked ventilation. Manufacturers have made the shell smooth in order to minimise drag, keeping air out in the process. The result is that you end up uncomfortable.

When Ash reviewed the BBB Tithon, for example, he said that the ventilation wasn't spectacular, and Dave said that Bolle's The One helmet was too hot when all the vents are covered (it's adjustable).

Giro Synthe MIPS 2 (1).jpg

Giro Synthe MIPS 2 (1).jpg

However, many of the latest generation aero road helmets come with good venting. The Giro Synthe (above), for example, the Kask Protone, and the Louis Garneau Course all feel very similar to standard road helmets in terms of the amount of air that gets to your head.

Trek Madone 2016 action  - 36

Trek Madone 2016 action - 36

When designing its new Ballista aero road helmet, Bontrager says that it created a thermal head form to evaluate the thermal efficiency of different helmets. The head form was covered with 36 thermal couples to determine the cooling properties of various designs and allow Bontrager to shape and position the vents most effectively.

Bontrager Ballista Helmet.jpeg

Bontrager Ballista Helmet.jpeg

As a result, the Ballista ended up with three vents in the front-centre, shaped to draw in air. Internal recessed channels are intended to manage airflow through the helmet and over the head, and exit ports at the back are designed to allow the air to escape easily so as not to increase drag.

Bontrager Ballista helmet - back

Bontrager Ballista helmet - back

Despite looking like it lacks ventilation, the Ballista feels cool in use, not too dissimilar to a normal road helmet.

Bell Star Pro helmet - detail

Bell Star Pro helmet - detail

Another option is to go with the Bell Star Pro which has a slider (above) in the top that allows you to open the vents for more cooling, and close them for improved aerodynamics.

Lazer Aeroshell.jpg

Lazer Aeroshell.jpg

Lazer takes a different approach with the Z1 helmet that we've reviewed here on road.cc (and others in the range). The Belgian brand allows you to use an Aeroshell (above) that can be either left off or added depending on whether you're prioritising cooling or aerodynamics (or protection from the weather conditions) on any given ride.

kask infinity helmet  - 4.jpg

kask infinity helmet - 4.jpg

Kask's Infinity is similar in that you get easy ventilation adjustment via a central sliding panel. It's easy to slide the cover back and forth with one hand on the move, though there is no small lever as there is on the Bell helmet.

Cannondale Cypher Aero Helmet.jpg

Cannondale Cypher Aero Helmet.jpg

Cannondale does something similar with its Cypher Aero

As a rule, aero road helmets are a little warmer than standard helmets. If you suffer with a hot head when riding or if it's a very hot day you might want to steer clear, but you’ll probably find a lot of the aero road helmets out there these days are comfortable in most conditions.

Weight

Here’s a quick roundup of the weights of the aero road helmets we’ve reviewed on road.cc (these are weights according to our scales rather than manufacturers' claimed weights).

Helmet Weight
Louis Garneau Course 297g
Bontrager Ballista 266g
Giro Synthe 223g
Kask Protone 250g
Bell Star Pro 300g
Spiuk Obuss 307g
BBB Tithon 280g
Bolle The One 340g
Cannondale Cypher Aero 285g
Kask Infinity 265g
Giant Rivet 299g

Giant claims a weight of under 250g for the new Pursuit aero helmet that we first saw at the Tour de France.

Giant Pursuit helmet  - 2.jpg

Giant Pursuit helmet - 2.jpg

Aero road helmets do tend to be a little heavier than standard helmets of a similar price, but not by much and there are exceptions. 

Price

Aero road helmets haven’t tended to be cheap in the past. The Kask Infinity is £220, for example, the Giro Synthe is £199.99 and, as mentioned above, Specialized’s S-Works Evade Team is £160.

However, there are less expensive options out there these days. The Bontrager Ballista is cheaper than most at £129.99, the Spiuk Obuss is £99.95, and the BBB Tithon is £79.95. 

Safety

Aero road helmets are made to meet the same safety standards as standard helmets .

So, should I buy an aero helmet?

Most aero road helmets are a little heavier and a little less vented than standard helmets, and they tend to be a little more expensive.

Trek Madone 2016 action  - 37_0.jpg

Trek Madone 2016 action - 37_0.jpg

On the flipside, you’ll to get a small reduction in drag. It’s a marginal gain but it might just make an important difference if you intend to go off the front and try your luck in a race.

Check out all of our helmet reviews here. For reviews on individual aero helmets, scroll up to the Weight section above and click on each model name.

Mat has worked for loads of bike magazines over 20+ years, and been editor of 220 Triathlon and Cycling Plus. He's been road.cc technical editor for eight years, testing bikes, fettling the latest kit, and trying out the most up-to-the-minute clothing. We send him off around the world to get all the news from launches and shows too. He has won his category in Ironman UK 70.3 and finished on the podium in both marathons he has run. Mat is a Cambridge graduate who did a post-grad in magazine journalism, and he is a past winner of the Cycling Media Award for Specialist Online Writer.

46 comments

Avatar
tritecommentbot [2268 posts] 1 year ago
0 likes

Got a Giro Synthe yesterday, weighed 249g in a size M.

 

Actually feels really heavy to me when on, maybe because my other hat is a Prolight around 169g. 

 

Get used to it I guess 

 

 

Avatar
Alessandro [144 posts] 1 year ago
0 likes

I'm soon to be moving to Edinburgh and have been considering for some time now whether to invest in an aero helmet, not for its alleged speed benefits but to keep my heid warm. Has anyone got any experience of helmets like the Lazer? Do they indeed keep your head warm in cold/wet weather or do you just end up sweating profusely underneath it?

Avatar
tritecommentbot [2268 posts] 1 year ago
4 likes
AST1986 wrote:

I'm soon to be moving to Edinburgh and have been considering for some time now whether to invest in an aero helmet, not for its alleged speed benefits but to keep my heid warm. Has anyone got any experience of helmets like the Lazer? Do they indeed keep your head warm in cold/wet weather or do you just end up sweating profusely underneath it?

 

No experience there, but consider adding a thin merino beanie and a windproof headband (to protect your ears from sub zero wind chill!) to your kit.

 

 

Avatar
Mo [2 posts] 1 year ago
8 likes

I know, I know, it's not supposed to be proper science, but I don't quite understand the point of these pseudo-scientific experiments. They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

I'm not saying you should do all of that in a road.cc helmet review; on the contrary, I think you should stay away from it as far as you can. If you can't do a scientific experiment, don't do one, because there will always be readers who don't get the difference -- and spend 200 Pounds on an aero helmet based on facts that simply aren't facts. The same goes for other magical numbers in cycling, like the stiffness of bike frames and wheels.

Just my humble opinion.

Avatar
Simon E [3072 posts] 1 year ago
4 likes

Similar to a buff, beanie etc, a cycling cap keeps the cold and rain/sleet/snow off your noggin and will prevent the precipitation hitting your face, either under a helmet or on its own. Highly recommended.

As for the aero benefits, once you sit up or take your hands off the bars to grab a gel/fiddle with your garmin/gesticulate at a driver then you've more than negated the tiny gain the hat might provide. Wear one if you like it but don't think it's faster than a vented lid in the real world.

Avatar
peted76 [687 posts] 1 year ago
7 likes

I've had an Evade helmet now for some time now..  almost every Sunday, one of the faster fella's teases me about how it should 'make all the difference'... (gentle teasing, no offence meant) - this guy looks like action man and rides like a boss, he's been in the 'A' riding group since I joined the club. 

One Sunday my 'B' group caught the 'A' group on the last five or six miles back to base.... I must have had a good day and he must have had a bad one as for some reason I found myself on his wheel when he attacked, as you do... he attacked, we dropped the group,  then I attacked and dropped him...  back at base I told him it wouldn' have happened if he'd had an aero helmet. 

The teasing stopped after that. 

Avatar
paulrattew [207 posts] 1 year ago
5 likes

I've got a Kask Infinity. Screw it being an aero helmet - it is my go to bad (wet / cold) weather helmet. I hate wearing anything more than a thin cap under my helmet, so being able to close the vents is great when it's really cold or when it is raining. Plus, I got it massively discounted.

Frankly, no matter how aero a helmet is, it is fighting a losing battle against the drag produced by my seemingly ever expanding waistline

Avatar
peted76 [687 posts] 1 year ago
0 likes
AST1986 wrote:

I'm soon to be moving to Edinburgh and have been considering for some time now whether to invest in an aero helmet, not for its alleged speed benefits but to keep my heid warm. Has anyone got any experience of helmets like the Lazer? Do they indeed keep your head warm in cold/wet weather or do you just end up sweating profusely underneath it?

A mate had a lazer with the aero shield and did find it too hot and sweaty, he changed to a kask becuase of it. 

Avatar
tritecommentbot [2268 posts] 1 year ago
4 likes
Mo wrote:

I know, I know, it's not supposed to be proper science, but I don't quite understand the point of these pseudo-scientific experiments. They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

I'm not saying you should do all of that in a road.cc helmet review; on the contrary, I think you should stay away from it as far as you can. If you can't do a scientific experiment, don't do one, because there will always be readers who don't get the difference -- and spend 200 Pounds on an aero helmet based on facts that simply aren't facts. The same goes for other magical numbers in cycling, like the stiffness of bike frames and wheels.

Just my humble opinion.

 

Nothing a simple caveat doesn't resolve.

 

Some major problems with this view.  The first is the obvious one, even manufacturers with wind tunnels can have holes picked in their studies. The second is that you're advocating a race to the bottom, which we have already, hence the predictable, copy-paste reviews. And thirdly, without any testing, you don't discover problems, which you can then go on and refer back to the manufacturer, or carry out more in-depth analysis.

 

More testing, and improve the quality of it incrementally. If sites like road.cc had focused on testing since their inception, they'd be far more thorough. 

Nice one guys for this article. Credit where credit's due.

 

 

Avatar
Windydog [67 posts] 1 year ago
1 like
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

Avatar
Yorkshire wallet [1323 posts] 1 year ago
2 likes

The cost of an aero helmet - £250

Moving up the Strava positions - priceless

Avatar
tritecommentbot [2268 posts] 1 year ago
2 likes
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

 

Avatar
Windydog [67 posts] 1 year ago
1 like

As they say in the article, every little helps.   I can only test it and see where I come in the standings?  Its a competitive advantage, until we all have them.

Avatar
part_robot [259 posts] 1 year ago
1 like

Yeah, people tend to scoff at 5W here and 10W there... but they tend not to appreciated that the differences to the rider (being a biological entity, not just a motor) are quite large. Take a look at your/a FTP curve to show how a small change in power makes a big difference in the time at which that power can be sustained. So 10W from a helmet, 10W from the tyres, 10W from the frame etc is something that really affects results... a lot. GCN did a great test on aero wheels showing just that.

Btw, if you're looking to get a Synth, Evade or Protone set your expectations about ventilation lower than the article suggests. They definitely aren't as cool as more traditional road helmets like the Mojito. So whilst I don't usually notice when giving it some welly - I'm more paying attention to my burning legs and my pain face - if you do reflect on them for a minute it's clear they are pretty hot.

Avatar
Eg3ftp1 [70 posts] 1 year ago
1 like
Mo wrote:

I know, I know, it's not supposed to be proper science, but I don't quite understand the point of these pseudo-scientific experiments. They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

I'm not saying you should do all of that in a road.cc helmet review; on the contrary, I think you should stay away from it as far as you can. If you can't do a scientific experiment, don't do one, because there will always be readers who don't get the difference -- and spend 200 Pounds on an aero helmet based on facts that simply aren't facts. The same goes for other magical numbers in cycling, like the stiffness of bike frames and wheels.

Just my humble opinion.

I think this sort of report is great, and just the sort of thing Road.cc should be doing more of. The real-world data aquired by Dave is the most scientific and useful of all the tests shown, as it shows  measureable difference for an actual rider doing typical speeds. He's using a power meter, so it doesn't matter whether he knows which helmet he's using, as long as he's not changing his body position with the helmet, which seems unlikely. If they could get 3-4 riders doing this sort of test, to clock up 100 or so laps, they could have some real confidence in the effect, and the info would be very useful when it comes to choosing new kit. 

How about doing this test with a team of 3-4 riders of different weights, for all reviewed kit which claims to offer a speed advantage: wheels, frames, skinsuits, as well as helmets? It would only be 1-2 hrs on the circuit with a bunch of you, swapping the kit over and recording what kit each of you is using for each of the laps, and you'd have the most convincing and useful info for prospective buyers on all of the internet?

Avatar
A2thaJ [68 posts] 1 year ago
0 likes
peted76 wrote:
AST1986 wrote:

I'm soon to be moving to Edinburgh and have been considering for some time now whether to invest in an aero helmet, not for its alleged speed benefits but to keep my heid warm. Has anyone got any experience of helmets like the Lazer? Do they indeed keep your head warm in cold/wet weather or do you just end up sweating profusely underneath it?

A mate had a lazer with the aero shield and did find it too hot and sweaty, he changed to a kask becuase of it. 

 

+1

I have the cover for a lazer helmet. As it only cost me an extra £16 to 'aero' my existing helmet, its not bad.... but it is hot. Fine for crits and TT's, but everyday use, or use in the summer, probably not suitable.

 

If you've already got that helmet, the shell add on is worth it. But to buy from scratch i'd look elsewhere.

Avatar
TeamExtreme [104 posts] 1 year ago
1 like
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

Avatar
freeewheelin [13 posts] 1 year ago
0 likes
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

Avatar
I am a human [32 posts] 1 year ago
1 like
AST1986 wrote:

I'm soon to be moving to Edinburgh and have been considering for some time now whether to invest in an aero helmet, not for its alleged speed benefits but to keep my heid warm. Has anyone got any experience of helmets like the Lazer? Do they indeed keep your head warm in cold/wet weather or do you just end up sweating profusely underneath it?

I got a Giro Air Attack for exactly this reason. It really does cut down on the horrible brain freeze you get on really cold days. It's not been too warm though, which is quite surprising. 

One thing that the article doesn't mention is that the Air Attack is noticeably quieter in terms of wind noise compared to my very vented other helmet. It really helps when keeping an ear out for traffic. 

Avatar
TeamExtreme [104 posts] 1 year ago
0 likes
freeewheelin wrote:
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

Well, for obvious editorial reasons there are precious few details on the experimental protocol or resulting data set, so your guess is as good as mine; all I can say is that the graph presented does not support the claim that "there's a measurable difference in the real world", it says that you've measured difference. To prove that the helmet was the cause of that difference is a very different prospect.

If you want to know more about the kind of experimental rigourousness that you have to employ to actually get meaningful data out of a test like this, have a read of this thread and ideally give it a go yourself.

http://forum.slowtwitch.com/forum/Slowtwitch_Forums_C1/Triathlon_Forum_F...

 

Avatar
Mo [2 posts] 1 year ago
1 like
freeewheelin wrote:
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

I think TeamExtreme summed up nicely what I was trying to say: it's not that I wouldn't love to see more scientific experiments on these topics, it's just that the overhead is probably too high. A semi-scientific method does not result in semi-significant results, it just results in no significance at all.

Again, things like randomization are important to look out for, but also, in this case, the power output of the rider. Given that the power-to-speed-relationship is not linear, an average power output does not tell you anything about the lap time, i.e., even under exactly the same test conditions and for the same average power output, you may end up with different lap times. How do you separate that from the aero-effect?

Talking about non-linear relationships, most riders will know that aero-efficiency is more important at higher speeds since the drag increases exponentially. However, you don't see that in the test results. According to the regression fit, the (absolute) savings are the same at high and low speeds, which does not indicate that the differences are caused by aerodynamic effects. Your absolute savings should be higher at higher speeds. I think if the author tries to perform an experiement in the field of aerodynamics, he/she should at least comment on things like that.

Avatar
tritecommentbot [2268 posts] 1 year ago
2 likes
Mo wrote:
freeewheelin wrote:
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

I think TeamExtreme summed up nicely what I was trying to say: it's not that I wouldn't love to see more scientific experiments on these topics, it's just that the overhead is probably too high. A semi-scientific method does not result in semi-significant results, it just results in no significance at all.

Again, things like randomization are important to look out for, but also, in this case, the power output of the rider. Given that the power-to-speed-relationship is not linear, an average power output does not tell you anything about the lap time, i.e., even under exactly the same test conditions and for the same average power output, you may end up with different lap times. How do you separate that from the aero-effect?

Talking about non-linear relationships, most riders will know that aero-efficiency is more important at higher speeds since the drag increases exponentially. However, you don't see that in the test results. According to the regression fit, the (absolute) savings are the same at high and low speeds, which does not indicate that the differences are caused by aerodynamic effects. Your absolute savings should be higher at higher speeds. I think if the author tries to perform an experiement in the field of aerodynamics, he/she should at least comment on things like that.

 

What is semi-scientific? You accused the author of pseudo science earlier, but talk in pseudo science. None of these critcisms are profound in the slightest and are all addressed in lab testing.

It's odd to me that you can't discern between a rigorous lab test and a real world test of a review site. A real world test merely has to follow the scientific method, and this test has. 

It will be as accurate as it is rigorous. It isn't very rigorous, as the author points out, so won't be very accurate. It simply gives us a data point that confirms manufacturers lab testing.

 

Trying to take down a reviewer for doing a quick real world test based on lab standards is irrational, and the mark of a pseud. 

Avatar
freeewheelin [13 posts] 1 year ago
0 likes
TeamExtreme wrote:
freeewheelin wrote:
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

Well, for obvious editorial reasons there are precious few details on the experimental protocol or resulting data set, so your guess is as good as mine; all I can say is that the graph presented does not support the claim that "there's a measurable difference in the real world", it says that you've measured difference. To prove that the helmet was the cause of that difference is a very different prospect.

If you want to know more about the kind of experimental rigourousness that you have to employ to actually get meaningful data out of a test like this, have a read of this thread and ideally give it a go yourself.

http://forum.slowtwitch.com/forum/Slowtwitch_Forums_C1/Triathlon_Forum_F...

 

 

Sorry, I wanted to know where the author deviated from the scientific method. Can you point out exactly where that was?

Avatar
flobble [116 posts] 1 year ago
1 like
I am a human wrote:
AST1986 wrote:

I'm soon to be moving to Edinburgh and have been considering for some time now whether to invest in an aero helmet, not for its alleged speed benefits but to keep my heid warm. Has anyone got any experience of helmets like the Lazer? Do they indeed keep your head warm in cold/wet weather or do you just end up sweating profusely underneath it?

I got a Giro Air Attack for exactly this reason. It really does cut down on the horrible brain freeze you get on really cold days. It's not been too warm though, which is quite surprising. 

One thing that the article doesn't mention is that the Air Attack is noticeably quieter in terms of wind noise compared to my very vented other helmet. It really helps when keeping an ear out for traffic. 

 

Similarly for me with Kask Infinity. I wear it on cold days and will open/close the vents according to my head temperature. It does make a difference.

Strangely, I've never felt too hot in it on warm days.

Avatar
TeamExtreme [104 posts] 1 year ago
0 likes
freeewheelin wrote:
TeamExtreme wrote:
freeewheelin wrote:
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

Well, for obvious editorial reasons there are precious few details on the experimental protocol or resulting data set, so your guess is as good as mine; all I can say is that the graph presented does not support the claim that "there's a measurable difference in the real world", it says that you've measured difference. To prove that the helmet was the cause of that difference is a very different prospect.

If you want to know more about the kind of experimental rigourousness that you have to employ to actually get meaningful data out of a test like this, have a read of this thread and ideally give it a go yourself.

http://forum.slowtwitch.com/forum/Slowtwitch_Forums_C1/Triathlon_Forum_F...

Sorry, I wanted to know where the author deviated from the scientific method. Can you point out exactly where that was?

As stated above, in drawing a conclusion that is not supported by the data.

Avatar
freeewheelin [13 posts] 1 year ago
0 likes
TeamExtreme wrote:
freeewheelin wrote:
TeamExtreme wrote:
freeewheelin wrote:
TeamExtreme wrote:
unconstituted wrote:
Windydog wrote:
Mo wrote:

They may produce some nice curves, but without a full assessment of all relevant variables, test randomization, a test rider who doesn't actually know which helmet he/she's wearing and adequate statistic analysis -- and no, a regression analysis is not an adequate statistical test -- the results have no meaning whatsoever.

Okay so how would you do it?  Variables - power meter and a loop course knock out placebo effect and weather.  What other variables are there for a real life regression analysis?  Willing listener here - please design me a test and a standard to validate these claims.

Put a robot in it, pin it to 200w on a flat indoor track maybe.   Or run a thousand sample test.  Is that science enough?  But then we couldn't afford the damn things for the R&D cost.  It's a lid, it works to an effect.

 

He's all mixed up. He doesn't see the difference between essential controlled lab testing that a manufacturer should do, and real world testing to see if on-the-road experience correlates to the manufacturer's resultant claims.

 

All review sites need to do is sincerely head out, as Mat and Dave have here, and get some data points and we can see for ourselves if things stack up.

 

I'm definitely with Mo on this one. You can't use "real world testing" as a synonym for "half-baked testing".

The fact of the matter is, without error bars on your set of data noone knows how rigorous the testing protocol is. It could very easily be the case that if Dave were to head out and do that exact same test again he'd get completely opposite results because the variability in the testing protocol massively outweighs the effect that he's trying to measure! 

Scientific methodology and statistical analyses exist for a reason and that is to ensure that the conclusions drawn are valid and supported by the data.

I agree, I'd love to see road.cc do more of this kind of thing, but the reality is, if it's not done properly then it's hardly worth doing at all.

 

Which parts of their testing were 'half-baked'. At what point did it deviate from the scientific method?

Well, for obvious editorial reasons there are precious few details on the experimental protocol or resulting data set, so your guess is as good as mine; all I can say is that the graph presented does not support the claim that "there's a measurable difference in the real world", it says that you've measured difference. To prove that the helmet was the cause of that difference is a very different prospect.

If you want to know more about the kind of experimental rigourousness that you have to employ to actually get meaningful data out of a test like this, have a read of this thread and ideally give it a go yourself.

http://forum.slowtwitch.com/forum/Slowtwitch_Forums_C1/Triathlon_Forum_F...

Sorry, I wanted to know where the author deviated from the scientific method. Can you point out exactly where that was?

As stated above, in drawing a conclusion that is not supported by the data.

 

In your view. In my view, it does.

That's a difference in opinion when drawing a conculsion which happens even in rigourous studies. 

The actual method here was scientific, unless you can point to a deviation, which I'm still waiting for.

Avatar
TeamExtreme [104 posts] 1 year ago
0 likes
freeewheelin wrote:

In your view. In my view, it does.

I'm not sure your view would be upheld by many people with even a cursory knowledge of statistics, but that's your prerogative.

freeewheelin wrote:

That's a difference in opinion when drawing a conculsion which happens even in rigourous studies. 

Science doesn't have much room for opinion, it's usually just the difference between those people who understand the data and those who don't.

freeewheelin wrote:

The actual method here was scientific, unless you can point to a deviation, which I'm still waiting for.

As stated previously, there is little information about the testing protocol used, but what has been presented does not support the conclusions drawn.

Avatar
racingcondor [233 posts] 1 year ago
1 like

I bought the Bontrager Ballista earlier this year, mostly bought it because it's a great fit for the price (the other really good fitting helmet I've had was an S-Works). Saved some cash and may have made myself marginally faster on the days I care.

Unfortunately 3 months later I trashed the helmet (and my collarbone) in a crash. Bontrager cover their helmets for a year under crash replacement so I'm getting a free replacement.

I'd prefer not to have needed it but for an aero helmet it's an excellent deal.

Avatar
BBB [455 posts] 1 year ago
1 like
part_robot wrote:

Yeah, people tend to scoff at 5W here and 10W there... but they tend not to appreciated that the differences to the rider (being a biological entity, not just a motor) are quite large. Take a look at your/a FTP curve to show how a small change in power makes a big difference in the time at which that power can be sustained. So 10W from a helmet, 10W from the tyres, 10W from the frame etc is something that really affects results... a lot. GCN did a great test on aero wheels showing just that.

Btw, if you're looking to get a Synth, Evade or Protone set your expectations about ventilation lower than the article suggests. They definitely aren't as cool as more traditional road helmets like the Mojito. So whilst I don't usually notice when giving it some welly - I'm more paying attention to my burning legs and my pain face - if you do reflect on them for a minute it's clear they are pretty hot.w

You own power output, weight and aerodynamics (flexibility) are almost entirely responsible for overall performance.

Your example of 30 watts (perhaps much more?) savings is based on numbers marketers put in your head. Almost all tests wind tunnels are based on unrealistic/irrelevant scenarios (setups etc.) manufacturing some meaningless numbers that get stuck in naive insecure consumers' heads. Those figures are then quoted endlessly on forums as if they were absolutely true.u

A series of indpenedent tests with average riders cycling outside at 20mph in a mix terrain would certainly reveal how little difference all the kit/gear actually makes.

 

P.S. I don't believe that the average speed of a typical amateur either riding solo or on a club ride has changed much in the last two or three decades despite all the technological (aero) advances. Has it?

 

 

Avatar
The _Kaner [1133 posts] 1 year ago
0 likes

I'm only after getting a Giro Air Attack...it was €59.23 from Amazon.

I can't attest to it's aero claims (yet) as I have still to wear it out on the bike. 

It is a fair bit heftier than my Scott Vanish R.

It is rather mushroom shaped on my petite heid.

It will, however, provide more coverage for my (newly acquired) shiny/hairless heid...the vagaries of getting a wee bit auld!

...it might 'shave' 10microseconds off my 40km times...or it might not...it might allow me to go that distance without having to 'emit' as many watts...or it might sit on a shelf for ever more after it's first outing has concluded...

Seat of the pants testing at it's best...

 

Pages