Benchmarking

[newdep]

right in the middle

1943 ms on at AMD 64 3200+

performance ratio:  0.8 (1.0 on Mac OS X, 1.83 GHz Intel Core 2 Duo)

[Joe]

1276 ms  =  pe:  0.6 (1.0 on Mac OS X, 1.83 GHz Intel Core 2 Duo)



Ran 5 times, all results within a range of 1276 - 1281 ms.

[cormullion]

Joe: would you like to swap your machine with mine? :)



edit: I found a supercomputer upstairs you might prefer:

Code Select Expand

12606 ms         ; pr: 5.0 ; Mac OS X 700 MHz PowerPC G3

that's from 2001, that machine...

[Joe]

Joe: would you like to swap your machine with mine? :)

The program reported the wrong hardware. It is actually:



2 x 3.2 GHz Quad-Core Intel Xeon



But you probably won't want it. It has some fingerprints and dust on it.

[cormullion]

Joe: would you like to swap your machine with mine? :)

The program reported the wrong hardware. It is actually:



2 x 3.2 GHz Quad-Core Intel Xeon



But you probably won't want it. It has some fingerprints and dust on it.

;)



That 'wrong hardware' is actually the reference machine used (ie lutz' machine is a "Mac OS X, 1.83 GHz Intel Core 2 Duo"). It would be cool to be able to find out easily what's running underneath, but I don't think there's a way.

[newdep]

Code Select Expand



34928 ms on a Nokia N810 armv61
performance ratio: 13.7



PS: well its 14 times smaller then a pc too.. so a small excuse is permitted ;-)

[pjot]

With the 64bit newLisp

newLISP v.10.1.1 64-bit on Linux IPv4, execute 'newlisp -h' for more info.

my computer gets this result:

1124 ms on a 2.2Ghz AMD Phenom(tm) 9550 Quad-Core Processor

performance ratio:  0.5 (1.0 on Mac OS X, 1.83 GHz Intel Core 2 Duo)

[xytroxon]

Code Select Expand



34928 ms on a Nokia N810 armv61
performance ratio: 13.7



PS: well its 14 times smaller then a pc too.. so a small excuse is permitted ;-)

My old country home net-surfer over 26400 baud phone line has got you all beat!



Intel Pentium 120 - P54CQS - 120MHz - cache (none) - 11.9 W

Win 95 (1996 BIOS copyright) upgraded to Win 98 and 16 GB HD in 1999, no USB, but has holes in the printed circuit board when USB hardware becomes more widely available ;0)



You've come a long way Intel baby since Win 95/98 "daze"!

Code Select Expand


!=             330 ms
$              440 ms
%              280 ms
&              380 ms
*              330 ms
+              280 ms
-              330 ms
/              270 ms
<              280 ms
<<             270 ms
<=             270 ms
=              280 ms
>              270 ms
>=             220 ms
>>             330 ms
NaN?           550 ms
^              390 ms
abs            330 ms
acos           490 ms
acosh          390 ms
add            320 ms
address        330 ms
amb            330 ms
and            220 ms
append         270 ms
apply          270 ms
args           390 ms
array          270 ms
array-list     270 ms
array?         390 ms
asin           980 ms
asinh          1100 ms
assoc          330 ms
atan           390 ms
atan2          380 ms
atanh          390 ms
atom?          330 ms
base64-dec     600 ms
base64-enc     440 ms
bayes-query    220 ms
bayes-train    770 ms
begin          270 ms
beta           610 ms
betai          600 ms
bind           390 ms
binomial       600 ms
bits           600 ms
case           440 ms
catch           60 ms
ceil           380 ms
char           940 ms
chop           330 ms
clean           50 ms
cond           280 ms
cons           270 ms
constant       220 ms
context        280 ms
context?       270 ms
copy           280 ms
cos            380 ms
cosh           500 ms
count          330 ms
cpymem         380 ms
crc32          170 ms
crit-chi2      760 ms
crit-z         500 ms
curry          160 ms
date           1820 ms
date-value     270 ms
debug          610 ms
dec            430 ms
def-new        330 ms
default        170 ms
define         270 ms
define-macro   330 ms
delete         1600 ms
det            330 ms
difference     330 ms
div            320 ms
do-until       330 ms
do-while       440 ms
doargs         330 ms
dolist         390 ms
dostring       380 ms
dotimes        390 ms
dotree         380 ms
dump           380 ms
dup            270 ms
empty?         330 ms
encrypt        330 ms
ends-with      490 ms
env            720 ms
erf            330 ms
error-event    280 ms
eval           270 ms
eval-string    440 ms
exists          60 ms
exp            430 ms
expand         280 ms
explode        330 ms
factor         330 ms
fft            490 ms
filter          60 ms
find           440 ms
find-all       330 ms
first          330 ms
flat           330 ms
float          980 ms
float?         280 ms
floor          440 ms
flt            330 ms
for            330 ms
for-all         50 ms
format         1700 ms
fv             330 ms
gammai         550 ms
gammaln        440 ms
gcd            220 ms
get-char       390 ms
get-float      490 ms
get-int        1320 ms
get-long       440 ms
get-string     380 ms
global         220 ms
global?        280 ms
if             270 ms
if-not         330 ms
ifft           500 ms
import         270 ms
inc            390 ms
index           50 ms
int            610 ms
integer?       220 ms
intersect      330 ms
invert         330 ms
irr            440 ms
join           330 ms
lambda?        280 ms
last           330 ms
last-error     440 ms
legal?         380 ms
length         440 ms
let            330 ms
letex          330 ms
letn           270 ms
list           280 ms
list?          220 ms
local          330 ms
log            380 ms
lookup         330 ms
lower-case     330 ms
macro?         220 ms
main-args      330 ms
map            330 ms
mat            330 ms
match          330 ms
max            380 ms
member         440 ms
min            330 ms
mod            330 ms
mul            390 ms
multiply       330 ms
name           440 ms
new            330 ms
nil?           380 ms
normal         490 ms
not            280 ms
now            2750 ms
nper           430 ms
npv            330 ms
nth            330 ms
null?          270 ms
number?        330 ms
or             390 ms
pack           490 ms
parse          390 ms
pmt            440 ms
pop            380 ms
pop-assoc      330 ms
pow            330 ms
pretty-print   440 ms
primitive?     270 ms
prob-chi2      500 ms
prob-z         330 ms
protected?     220 ms
push           380 ms
pv             330 ms
quote          330 ms
quote?         220 ms
rand           660 ms
random         380 ms
randomize      330 ms
read-expr      440 ms
ref            390 ms
ref-all        330 ms
regex          440 ms
regex-comp     490 ms
replace        220 ms
rest           440 ms
reverse        380 ms
rotate         390 ms
round          1430 ms
seed           110 ms
select         380 ms
sequence       330 ms
series         270 ms
set            280 ms
set-locale     990 ms
set-ref        380 ms
set-ref-all    330 ms
setf           170 ms
setq           330 ms
sgn            330 ms
sin            440 ms
sinh           430 ms
slice          380 ms
sort            60 ms
source         3460 ms
sqrt           330 ms
starts-with    550 ms
string         7630 ms
string?        440 ms
sub            330 ms
swap           390 ms
sym            380 ms
symbol?        390 ms
symbols        490 ms
sys-error      490 ms
sys-info       330 ms
tan            440 ms
tanh           390 ms
throw           50 ms
throw-error    330 ms
time           9450 ms
time-of-day    3510 ms
title-case     390 ms
transpose      440 ms
trim           220 ms
true?          330 ms
unify          380 ms
unique         330 ms
unless         330 ms
unpack         440 ms
until          330 ms
upper-case     330 ms
uuid           2200 ms
when           270 ms
while          440 ms
write-buffer   440 ms
write-line     440 ms
xml-parse      380 ms
xml-type-tags  330 ms
zero?          280 ms
|              440 ms
~              1590 ms
127470 ms
performance ratio: 50.0 (1.0 on Mac OS X, 1.83 GHz Intel Core 2 Duo)



Add another 5 to the performance ratio for printing output to screen instead of to a file...



The funny thing is, that at work, I can't read web pages any faster on a newer machine ;o)



Lookup your evil, power hungry, global warming Intel beast chip here:

http://www.processor-comparison.com/power.html">//http://www.processor-comparison.com/power.html



-- xytroxon

[Elica]

A little bit too late, but here are my two cents on the topic of benckmarks (I did a lot for the Lhogho compiler).



1. I think it is impossible to write a satisfactory one-liner for benchmarking.



2. A true benchmark does not produce a scalar (i.e. a single value), but a vector - i.e. there are many individual performance values, measuring different aspects. For example, scores are for integer arithmetics, for fp arithmetics, for string manipulation, for memory management, for passing parameters, for searching local variables, etc. etc.



3. Except for a vector of many specialized mini-benchmarks, there is an option to use a more complex system, which measures a predefined set of functions. For example, a test for calculation a recursive math function spends time for real calculaitons, for passing parameters, for low-level function invokation, and so on.



4. Simple/short benchmarks can be seriously affected by the Operating system (especially what it is doing in the background while you do the becnmark)



5. The way a code is compiled can also lead to drastically different results. For example, if newLisp is compiled with and without optimizations, this may lead to noticeable difference in performance.



6. What does one-line mean in a language where newlines are just whitespaces?



7. For a one-liner that measure integer arithmetics you can try the 3x+1 problem.