The forum for all of your Freebasic needs!

Moderators: Pete, Mods

dinokilla
Coder
Posts: 12
Joined: Mon Jan 07, 2008 6:56 pm

so I'm making some crappy little math tutor programs for a younger sibling, I thought doing operations with decimals would be pretty straight forward but I come up with odd answers occasionaly. The answer are always right but once in awhile there will be an extra 3 digits on the end of the answer (typically 001,002,999). This is causing me problems when verifying the answers. It will sometimes produce correct results for a long period of time and then all of a sudden I get a strange result like that. I can get around it converting it to a string and back I think but is there a better way to do this or am I doing something wrong? Thanks...

oohhhhh btw...in some of the operations I want the decimal places to line up so I use a random 5 digit integer multiplied by .001 using CSNG into a single...could that be the problem?

I want to add numbers like 21.103+13.353 but sometimes the result of this will be 34.455 why would it round it down like that? it should be 34.456

Codemss
Veteran
Posts: 124
Joined: Sun Jun 24, 2007 6:49 am
Location: Utrecht, The Netherlands
Contact:
Yeah. This is very weird and it sucks. I have it too. Sometimes, If I add 0.2 to 0.2 I get 0.399999999999999 and that kind of things.
You should be able to get rid of this. You may round of on some decimal.
I dont know what's the best way to do that. Don't directly use CINT or INT (at least, not for this job I mean, ) if you want some decimals left .
Well, succes. If you find something on it, I'd be interested in it.
Hope you like it. Send some feedback if you want: <a href="mailto:basicallybest@live.nl">Mail me</a>
Codemss, before known as RubyNL

dinokilla
Coder
Posts: 12
Joined: Mon Jan 07, 2008 6:56 pm
even when I try to convert it to a string and then chop off the end and convert it back to single it still does that occasionaly. Most of the time it works fine but for whatever weird reason it will do that, the strangest part to me is sometimes there will be no carry involved, like the last digit of number1 is 4 and the last digit of number2 is 3, when added together you would expect 7 yet sometimes it will come up with 6!!!! I don't know any way to do it with 100% accuracy.

Nodtveidt
Veteran
Posts: 826
Joined: Sun Jul 25, 2004 4:24 am
Contact:
It's known as a BCD representation error, and there's no way around it. What you should do instead is used fixed point numbers.

dinokilla
Coder
Posts: 12
Joined: Mon Jan 07, 2008 6:56 pm
ok how do I go about doing that?

 nevermind I get it, is there any reason I couldn't use long's for that? thanks I never woulda thought of that!

dinokilla
Coder
Posts: 12
Joined: Mon Jan 07, 2008 6:56 pm
Wow what a huge pain in the butt just to do simple decimal addition!!!

Mentat
Veteran
Posts: 409
Joined: Tue Aug 07, 2007 3:39 pm
Location: NC, US
Use large integers, then divide by a magnitude or two.

Code: Select all

``````'some function that gets a random integer between two others
declare function randInt(low as integer, high as integer) as integer

'integers you want to work with
dim a as integer
dim b as integer

'some multiple of ten
const scale=100

a=randInt(1,1000)
b=randInt(1,1000)

print "Here's a number: ";a/scale
print "Here's another number: ";b/scale
print "Here's the sum: ";(a+b)/scale

sleep

end

function randInt(low as integer, high as integer) as integer
return rnd*(low-high)+low
end function
``````
For any grievances posted above, I blame whoever is in charge . . .

dinokilla
Coder
Posts: 12
Joined: Mon Jan 07, 2008 6:56 pm
run your code but give it a random seed (randomize timer) and run it a few times and you will see the problem I am talking about...

Mentat
Veteran
Posts: 409
Joined: Tue Aug 07, 2007 3:39 pm
Location: NC, US
Oh, I see what you mean. Well, I just happen to have a solution:

Code: Select all

``````randomize timer

'some function that gets a random integer between two others
declare function randInt(low as integer, high as integer) as integer

'integers you want to work with
dim a as integer
dim b as integer

'some multiple of ten
const scale=100

a=randInt(1,1000)
b=randInt(1,1000)

print "Here's a number: ";
print using "###.##";a/scale
print "Here's another number: ";
print using "###.##";b/scale
print "Here's the sum: ";
print using "###.##";(a+b)/scale

sleep
``````
For any grievances posted above, I blame whoever is in charge . . .

dinokilla
Coder
Posts: 12
Joined: Mon Jan 07, 2008 6:56 pm
hehehe, that wouldn't work either because it will just print it in the correct format but if you ever had to compare the answer the actual number might still be 3.230000000000000001 but I got it figured out for what I needed it for. I just make sure that I'm always using an integer for everything so it works great. Thanks for your help though!