# 2007-11-27: Comparison of numerical results on various Matlab # versions, CPUs and operating systems. The test-case uses pdepe # and fft, besides some basic array manipulation. # # OS=gentoo means linux kernel >2.6.20, glibc 2.5, gcc 4.1.x # OS=rhel5 means RedHat Enterprise 5: kernel 2.6.18.x, glibc 2.5. # # 64|32 bit refers to the kernel's support for 64bit programs, # and if these are supported then the OS is a 64bit-capable one # and the 64bit Matlab is used. # # A line of shell, used to run the example file: # for n in 5 4 3 1 ; do # matlab-7.$n -nojvm -nosplash -r 'ml_numres_example; exit' # done # First, ordered by system and versions. # CPU 64|32bit OS MLvers output Athlon-XP 32 gentoo 7.5 2.095668e-11, 4.166408e-15 Xeon-Core2 64 gentoo 7.5 2.095668e-11, 4.166408e-15 Xeon-Core2 64 gentoo 7.3 2.095661e-11, 4.100776e-15 Xeon-Core2 64 gentoo 7.4 2.095661e-11, 4.100776e-15 Xeon-Core2 64 gentoo 7.1 2.095664e-11, 4.154064e-15 Xeon-P4 64 gentoo 7.5 2.095668e-11, 4.166408e-15 Xeon-P4 64 gentoo 7.4 2.095661e-11, 4.100776e-15 Xeon-P4 64 gentoo 7.3 2.095661e-11, 4.100776e-15 Xeon-P4 64 gentoo 7.1 2.095664e-11, 4.154064e-15 Athlon64-X2 64 gentoo 7.5 2.095624e-11, 3.829428e-15 Athlon64-X2 64 gentoo 7.4 2.095661e-11, 4.100776e-15 Athlon64-X2 64 gentoo 7.3 2.095661e-11, 4.100776e-15 Athlon64-X2 64 gentoo 7.1 2.095664e-11, 4.154064e-15 Opteron 64 rhel5 7.5 2.095624e-11, 3.829428e-15 Opteron 64 rhel5 7.4 2.095661e-11, 4.100776e-15 Opteron 64 rhel5 7.3 2.095661e-11, 4.100776e-15 Opteron 64 rhel5 7.1 2.095664e-11, 4.154064e-15 Pentium4 32 gentoo 7.5 2.095668e-11, 4.166408e-15 Pentium4 32 gentoo 7.4 2.095661e-11, 4.100776e-15 Pentium4 32 gentoo 7.3 2.095661e-11, 4.100776e-15 Pentium4 32 gentoo 7.1 2.095664e-11, 4.154064e-15 Athlon64 64 rhel5 7.5 2.095624e-11, 3.829428e-15 Athlon64 64 rhel5 7.4 2.095661e-11, 4.100776e-15 Athlon64 64 rhel5 7.3 2.095661e-11, 4.100776e-15 Athlon64 64 rhel5 7.1 2.095664e-11, 4.154064e-15 # Now, the same lines, sorted on the last field to group similar # numerical behaviours. # CPU 64|32bit OS MLvers output Athlon64 64 rhel5 7.5 2.095624e-11, 3.829428e-15 Athlon64-X2 64 gentoo 7.5 2.095624e-11, 3.829428e-15 Opteron 64 rhel5 7.5 2.095624e-11, 3.829428e-15 Athlon64 64 rhel5 7.3 2.095661e-11, 4.100776e-15 Athlon64 64 rhel5 7.4 2.095661e-11, 4.100776e-15 Athlon64-X2 64 gentoo 7.3 2.095661e-11, 4.100776e-15 Athlon64-X2 64 gentoo 7.4 2.095661e-11, 4.100776e-15 Opteron 64 rhel5 7.3 2.095661e-11, 4.100776e-15 Opteron 64 rhel5 7.4 2.095661e-11, 4.100776e-15 Pentium4 32 gentoo 7.3 2.095661e-11, 4.100776e-15 Pentium4 32 gentoo 7.4 2.095661e-11, 4.100776e-15 Xeon-Core2 64 gentoo 7.3 2.095661e-11, 4.100776e-15 Xeon-Core2 64 gentoo 7.4 2.095661e-11, 4.100776e-15 Xeon-P4 64 gentoo 7.3 2.095661e-11, 4.100776e-15 Xeon-P4 64 gentoo 7.4 2.095661e-11, 4.100776e-15 Athlon64 64 rhel5 7.1 2.095664e-11, 4.154064e-15 Athlon64-X2 64 gentoo 7.1 2.095664e-11, 4.154064e-15 Opteron 64 rhel5 7.1 2.095664e-11, 4.154064e-15 Pentium4 32 gentoo 7.1 2.095664e-11, 4.154064e-15 Xeon-Core2 64 gentoo 7.1 2.095664e-11, 4.154064e-15 Xeon-P4 64 gentoo 7.1 2.095664e-11, 4.154064e-15 Athlon-XP 32 gentoo 7.5 2.095668e-11, 4.166408e-15 Pentium4 32 gentoo 7.5 2.095668e-11, 4.166408e-15 Xeon-Core2 64 gentoo 7.5 2.095668e-11, 4.166408e-15 Xeon-P4 64 gentoo 7.5 2.095668e-11, 4.166408e-15 # `We' already knew that numerical differences *can* arise due # to different compiler options, or different code being chosen # for different CPUs. It was assumed that Matlab would be # designed to provide similar results for its own functions, # independent of the CPU type, even at expense of some optimisation. # # Matlab uses FFTW: this is known for doing clever things to increase # speed but lose some accuracy. Is it *just* the FFT part that's # making the difference? # Tried looking at just the time-domain values (i_td) that are an # intermediate step in the calculation. These also revealed # differences, with the same pair being similar for the one host # tried (`Opteron'). # # The groupings of results show that there is no consistent effect of # a single parameter of CPU, 64|32 or OS, on the results. # Only Matlab version 7.5 showed variations between different systems; # this divided the systems into AMD's 64bit Opteron and Athlon64, # against any Intel or the older 32bit Athlon-XP. # Matlab versions 7.3 and 7.4 gave the same results as each other, # consistent over all systems tried. # Matlab version 7.1 was consistent between different systems, but # differed from all of the other versions. # # This particular case isn't a practical problem. More, I'm puzzled # by whether one really should expect such variation. # end # The example script. cat <