Yes, it was noticeable as this was a feature built into the main instruction set of the Athlon range so was taken advantage of during all CPU operations. Obviously all the other system architecture did effect the overall speed, so the fact that both Pentium 3's and Athlon's were using slow 100MHz and 133MHz DIMMS at the time wouldn't have helped.
But interestingly before the Athlon, AMD CPUs were actually slower than 1 floating point operation per clock cycle, compared to Intel which were always 1 measures as 1 per cycle, so before the Athlon range the AMD CPU's were slower than Intel CPU's on a clock for clock basis.
The introduction of the first Athlon with double the operations per clock cycle was the turning point for AMD and the point when AMD CPU's started to gain popularity. My first AMD CPU was a 400MHz K3 CPU in a laptop, so that would actually of had a MFLOP rating under 400MFLOPS, whereas my next was an Athlon 900MHz PC I had at work which would of had 1800MFLOPS.