Pete's QBASIC Site

VAL function problem

2009-12-11T01:33:49-05:00

You can use Double values. When the total is done, use a string. Single values may not work if they are over 7 digit numbers:

sum$ = LTRIM$(STR$(TOTAL#))

'Use INSTR to find the decimal point place:

position% = INSTR(sum$, ".") + 2 ' for 2 decimal places

result$ = MID$(sum$, 1, position%)

PRINT result$

Statistics: Posted by burger2227 — Fri Dec 11, 2009 1:33 am

VAL function problem

2009-12-11T00:27:14-05:00

burger,
You are correct. I changed my code and it corrected my VAL issue. But that brought about another problem, when I do the
TOTAL# = TOTAL# + value ---- it returns TOTAL as a Double value, but brings me right back to my original problem that I had. 9889.89 turns into 9889.8896484375 for some reason...

My total will be in the millions with a decimal, do I need to define it as a Double?

Crap.. this can't be this hard!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
All I want to add is add large numbers together....

Statistics: Posted by mrjaybov — Fri Dec 11, 2009 12:27 am

I see what happened...

2009-12-11T00:07:09-05:00

You are using Double variables which creates the longer value accuracy.

num$ = "9889.98"

value = VAL(num$) ' Single value is accurate

TOTAL# = TOTAL# + value ' returns TOTAL as a Double value

PRINT value, TOTAL#

Certain values, not all, will do that. If you are adding a lot of values, you can convert the results back to a string to get rid of the extra decimal places.

Or you can display the trimmed results with:

PRINT USING " #############,.##"; TOTAL#

PRINT USING will also round the decimal places, but that is just for show. The actual value does not change.

Statistics: Posted by burger2227 — Fri Dec 11, 2009 12:07 am

VAL function problem

2009-12-10T22:56:46-05:00

Most likely due to the fact that there's no possible 32 bit pattern for 9889.89, so it settles with the closest that it can find: 9889.8899999. Some decimal numbers (like 0.1 IIRC) are repeating decimals in binary, so it would take an infinite amount of bits to be right. Since that's not possible (we use 32 bits or 64 bits for floating point numbers), the computer approximates to the nearest possible number it can find. My guess is that this is one of those numbers.

Statistics: Posted by Mentat — Thu Dec 10, 2009 10:56 pm

VAL function problem

2009-12-10T22:15:58-05:00

I am pulling the number from a comma delimited file by using the MID$ command, AMT$ = MID$(data$,2,7) then all I do is add it to a totals variable like this: Total# = Total# + VAL(AMT$). Funny thing is, it only happens once in a while, there's no rhyme or reason. When I print out the string AMT$ it equals 9889.89, once it convert it using VAL(AMT$) and print I get 9889.8899999. Could it be a variable issue?

Statistics: Posted by mrjaybov — Thu Dec 10, 2009 10:15 pm

VAL function problem

2009-12-10T19:05:13-05:00

I can't duplicate the error. Try printing the string and VAL return.

Are you using variables like: value = VAL(number$) ?

Undefined numerical variables default as Single, but it seems like a Double problem.

Statistics: Posted by burger2227 — Thu Dec 10, 2009 7:05 pm

VAL function problem

2009-12-10T17:20:52-05:00

I have a string that equals 9889.98 when I use the VAL function to convert it to a number I get 9889.979999999 Why is that? I'm converting lots of numbers this way and I always get one or two that does this.. It's throwing my totals off.

Statistics: Posted by mrjaybov — Thu Dec 10, 2009 5:20 pm