"Unknown error-can't save file" FBC dies on compil

If you have questions about any aspect of QBasic programming, or would like to help fellow programmers solve their problems, check out this board!

Moderators: Pete, Mods

Post Reply
dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

"Unknown error-can't save file" FBC dies on compil

Post by dbish » Thu May 05, 2005 1:14 am

I have been working on a major series of data manipulation Freebasic programs. Most are single purpose and then I move on taking snippets of code from program to next. On my latest I have been developing a major routine which worked fine until I made one change and now FB IDE freezes when I try to compile. My working routine had the following DIM statements (just array DIMs shown):

OPTION BASE 1

DECLARE SUB TICKERDO (FIRSTTICKERROW AS INTEGER, LASTTICKERROW AS INTEGER)

DEFINT A-Z
DIM OPTIONLINE AS STRING
DIM SHARED WorkingRawArray(1 TO 250000, 1 TO 19) AS STRING
DIM SHARED WorkingFinalArray_OI(1 TO 25000, 1 TO 16) AS STRING
DIM SHARED WorkingFinalArray_VL(1 TO 25000, 1 TO 16) AS STRING
DIM SHARED WorkingLineArray(25000) as STRING
DIM SHARED KILLArray(1000) AS STRING
DIM SHARED FINALROWNUMBER_KILL AS INTEGER
DIM SHARED CalcArray_VL(250000) AS DOUBLE
DIM SHARED CalcArray_OI(250000) AS DOUBLE
DIM TKR_PREV(5000,2) as STRING ' <== Note this line will change
DIM TKR_NEW(5000,4) as STRING
DIM k AS INTEGER

I have re-thought my manipulations and now want to change the above noted line to
DIM TKR_PREV(5000) as STRING

I have made all the corresponding changes in my logic so the code looks for a single dimension array. Now, when I compile (using FBIDE v0.3.3a) I get the following errors:

First, I get a Windows error box (I am using Windows XP) which says:
----------------------------------------------------------------------------------------------------------------------
fbc.exe has encountered a problem and needs to close. We are sorry for the inconvenience.
----------------------------------------------------------------------------------------------------------------------

Then I get a error message from FBIDE which says:

----------------------------------------------------------------------------------------------------------------------
COMPILE ERROR
Unknown error. Suggestions:
you didn't save file as .bas
some library/include header is not linked
On Line:
----------------------------------------------------------------------------------------------------------------------

That?s it. If I change the dimensions on that array to any 2 DIM array (even 5000,1) it compiles fine. I have a .bas name on the code and I am not using any libraries or include headers. I have used many other 1 dimensional arrays without problem. I tried this problem on another machine and got exactly same error.

Anybody have any ideas?

Thanks

dbish

Z!re
Veteran
Posts: 887
Joined: Wed Aug 04, 2004 11:15 am

Post by Z!re » Thu May 05, 2005 5:33 am

The code requires atleast ~152MB free memory to work.. you got that?


Post the full souce, as it's ost likely something else making the error.
I have left this dump.

dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

Post by dbish » Thu May 05, 2005 8:45 am

I have plenty of memory (768 Mb installed - won't compile with even minimal other apps open).

Here is complete code:

------------------------------------------------------------------------------------------------------------------------------------------------------------

OPTION BASE 1

DECLARE SUB TICKERDO (FIRSTTICKERROW AS INTEGER, LASTTICKERROW AS INTEGER)

DEFINT A-Z
DIM OPTIONLINE AS STRING
DIM SHARED WorkingRawArray(1 TO 250000, 1 TO 19) AS STRING
DIM SHARED WorkingFinalArray_OI(1 TO 25000, 1 TO 16) AS STRING
DIM SHARED WorkingFinalArray_VL(1 TO 25000, 1 TO 16) AS STRING
DIM SHARED WorkingLineArray(25000) as STRING
DIM SHARED KILLArray(1000) AS STRING
DIM SHARED FINALROWNUMBER_KILL AS INTEGER
DIM SHARED CalcArray_VL(250000) AS DOUBLE
DIM SHARED CalcArray_OI(250000) AS DOUBLE
DIM TKR_PREV(5000) as STRING
DIM TKR_NEW(5000,3) as STRING
DIM k AS INTEGER
DIM FINALROWNUMBER AS INTEGER
DIM FINALROWNUMBER_NEW AS INTEGER
DIM FINALROWNUMBER_PREV AS INTEGER
DIM TKR_NEW_START as INTEGER
DIM TKR_PREV_START as INTEGER
DIM CHAR1 AS STRING
DIM FILENAMECURRENT AS STRING
DIM SHARED FIRSTTICKERROW AS INTEGER
DIM SHARED LASTTICKERROW AS INTEGER
DIM CL AS STRING
DIM var_DATE as STRING
DIM var_DAY as STRING
DIM var_MONTH as STRING
DIM var_YEAR as STRING
DIM SHared DATESTRING as STRING

SHELL "dir E:\Optdata\OI\*.csv /b > c:\temp2\TKR_PREV.csv"
i = 0
OPEN "C:\Temp2\TKR_PREV.csv" FOR INPUT AS #5
DO WHILE NOT EOF(5)
i = i + 1
LINE INPUT #5, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #5
FINALROWNUMBER_PREV% = i
OPEN "C:\Temp2\TKR_PREV.csv" FOR INPUT AS #5
FOR i = 1 TO FINALROWNUMBER_PREV%
INPUT #5, TKR_PREV$(i)
NEXT i
CLOSE #5
For i = 1 to FINALROWNUMBER_PREV%
TKR_PREV$(i)=LEFT$(TKR_PREV(i),LEN(TKR_PREV(i))-7)
Next i
KILL "c:\temp2\TKR_PREV.csv"
OPEN "c:\temp2\TKR_PREV.csv" FOR APPEND as #5
For i = 1 to FINALROWNUMBER_PREV%
PRINT #5, TKR_PREV$(i)
Next i
CLOSE #5

OPEN "C:\temp2\TKR.CSV" FOR APPEND AS #10
CLOSE #10
KILL "C:\temp2\TKR.CSV"
OPEN "C:\temp2\TKR.CSV" FOR APPEND AS #10

CLS
var_DATE$=DATE$
var_DAY$=MID$(var_DATE$,4,2)
var_MONTH$=LEFT$(var_DATE$,2)
var_YEAR$=RIGHT$(var_DATE$,4)
DATESTRING$=var_YEAR$+var_MONTH$+var_DAY$
'FILENAMECURRENT$ = "E:\OptData\Daily\"+DATESTRING$+".csv"
FILENAMECURRENT$ = "E:\OptData\Daily\20060315.csv"
DATESTRING$=MID$(FILENAMECURRENT$,18,8)


i = 0
OPEN "C:\Temp2\KILL_TICKER.txt" FOR INPUT AS #5
DO WHILE NOT EOF(5)
i = i + 1
LINE INPUT #5, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #5
FINALROWNUMBER_KILL% = i
OPEN "C:\Temp2\KILL_TICKER.txt" FOR INPUT AS #5
FOR i = 1 TO FINALROWNUMBER_KILL%
INPUT #5, KILLArray$(i)
NEXT i
CLOSE #5

i = 0
OPEN FILENAMECURRENT$ FOR INPUT AS #1
DO WHILE NOT EOF(1)
i = i + 1
LINE INPUT #1, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #1
FINALROWNUMBER% = i
OPEN FILENAMECURRENT$ FOR INPUT AS #1
FOR i = 1 TO FINALROWNUMBER%
FOR j = 1 TO 19
INPUT #1, WorkingRawArray$(i, j)
NEXT
NEXT
CLOSE #1

FIRSTTICKERROW% = 1
FOR i = 1 TO FINALROWNUMBER%
IF i = FINALROWNUMBER% THEN
LASTTICKERROW% = i
CALL TICKERDO(FIRSTTICKERROW%, LASTTICKERROW%)
EXIT FOR
END IF
IF WorkingRawArray$(i, 1) <> WorkingRawArray$(i + 1, 1) THEN
LASTTICKERROW% = i
CALL TICKERDO(FIRSTTICKERROW%, LASTTICKERROW%)
FIRSTTICKERROW% = LASTTICKERROW% + 1
END IF
NEXT i
CLOSE #10


OPEN "E:\OptData\SUMMARY_"+DATESTRING$+".csv" FOR APPEND AS #3

' TKR_PREV should still be open form beginning of run

'OPEN "c:\temp2\TKR_PREV.csv" FOR INPUT AS #2
'DO WHILE NOT EOF(1)
' i = i + 1
' LINE INPUT #1, OPTIONLINE$
' CHAR1$ = LEFT$(OPTIONLINE$, 1)
' IF CHAR1$ = "" THEN EXIT DO
'LOOP
'CLOSE #2
'FINALROWNUMBER_PREV% = i
'OPEN "c:\temp2\TKR_PREV.csv" FOR INPUT AS #2
'FOR i = 1 TO FINALROWNUMBER_PREV%
' INPUT #2, TKR_PREV(i)
' NEXT
'NEXT
'CLOSE #2

OPEN "c:\temp2\TKR.csv" FOR INPUT AS #2
DO WHILE NOT EOF(1)
i = i + 1
LINE INPUT #1, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #2
FINALROWNUMBER_NEW% = i
OPEN "c:\temp2\TKR.csv" FOR INPUT AS #2
FOR i = 1 TO FINALROWNUMBER_NEW%
FOR j = 1 TO 3
INPUT #2, TKR_NEW(i, j)
NEXT
NEXT
CLOSE #2
If FINALROWNUMBER_PREV% >= FINALROWNUMBER_NEW% then
TICKER_NEW_START%=0
Else
TICKER_NEW_START%=1
END IF

IF TICKER_NEW_START=0 THEN
FOR i=1 to FINALROWNUMBER_PREV%
For j= 1 to FINALROWNUMBER_NEW%
If TKR_PREV$(i)=TKR_NEW(j,1) then
If ((val(TKR_NEW$(i, 2))/val(TKR_PREV$(j,2))-1))<-.15 then
PRINT #3,"SPLIT?,"+TKR_NEW(i, 1)+","+TKR_NEW$(i, 2)+","+TKR_PREV$(j,2)
GOTO NEXTI_P1
END IF
END IF
NEXT j
PRINT #3,"MISSING?"+TKR_PREV$(i)+",--,--"
NEXTI_P1:
NEXT i
FOR i=1 to FINALROWNUMBER_NEW%
For j= 1 to FINALROWNUMBER_PREV%
If TKR_NEW(i, 1)=TKR_PREV(j,1) then
GOTO NEXTI_P2
END IF
NEXT j
PRINT #3,"NEW?,"+TKR_NEW(i, 1)+",--,--"
NEXTI_P2:
NEXT i
ELSE
FOR i=1 to FINALROWNUMBER_NEW%
For j= 1 to FINALROWNUMBER_PREV%
If TKR_NEW(i, 1)=TKR_PREV(j,1) then
If ((val(TKR_NEW$(i, 2))/val(TKR_PREV$(j,2))-1))<-.15 then
PRINT #3,"SPLIT?,"+TKR_NEW(i, 1)+","+TKR_NEW$(i, 2)+","+TKR_PREV$(j,2)
GOTO NEXTI_N1
END IF
END IF
NEXT j
PRINT #3,"NEW?"+TKR_NEW(i, 1)+",--,--"
NEXTI_N1:
NEXT i
FOR i=1 to FINALROWNUMBER_PREV%
For j= 1 to FINALROWNUMBER_NEW%
If TKR_PREV$(i)=TKR_NEW(j,1) then
GOTO NEXTI_N2
END IF
NEXT j
PRINT #3,"MISSING?,"+TKR_PREV$(i)+",--,--"
NEXTI_N2:
NEXT i
END IF

END

SUB TICKERDO (FIRSTTICKERROW%, LASTTICKERROW%)
DIM FILENAMESAVE_OI AS STRING
DIM FILENAMESAVE_VL AS STRING
DIM CHAR1 AS STRING
DIM CHAR2 AS STRING
DIM CHAR3 AS STRING
DIM CHAR4 AS STRING
DIM CHAR5 AS STRING
DIM CHARLEN AS INTEGER
DIM BID AS SINGLE
DIM ASK AS SINGLE
DIM CSPRICENUM AS SINGLE
DIM TICKER AS STRING
DIM CSPRICE AS STRING
DIM DATERAW as STRING
DIM DATALEN as Integer
DIM DATADATE_N as DOUBLE
DIM DATADATE_S AS STRING
DIM SLASHCOUNT as INTEGER
DIM SLASHARRAY(2) as INTEGER
DIM MONTH_S as STRING
DIM DAY_S as STRING
DIM YEAR_S as STRING
DIM DAY_N as INTEGER
DIM MONTH_N as INTEGER
DIM YEAR_N as INTEGER
DIM OI AS SINGLE
DIM VOL AS SINGLE
DIM CUMCALL_OI_N AS DOUBLE
DIM CUMCALL_OI_S AS STRING
DIM CUMPUT_OI_N AS DOUBLE
DIM CUMPUT_OI_S AS STRING
DIM CUMCALL_VL_N AS DOUBLE
DIM CUMCALL_VL_S AS STRING
DIM CUMPUT_VL_N AS DOUBLE
DIM CUMPUT_VL_S AS STRING
DIM PUTCALL_OI_N as Single
DIM PUTCALL_OI_S as STRING
DIM PUTCALL_VL_N as Single
DIM PUTCALL_VL_S as STRING
DIM MKTCAP_N as SINGLE
DIM MKTCAP_S as STRING
DIM MKTCAP_FacA_N as Single
DIM MKTCAP_FacA_S as String
DIM MKTCAP_FacB_N as Single
DIM MKTCAP_FacB_S as String
DIM MKTCAP_FacC_N as Single
DIM MKTCAP_FacC_S as String
DIM PC_SD_OI_N as SINGLE
DIM PC_SD_OI_S as STRING
DIM PC_SD_OI_FacA_N as Single
DIM PC_SD_OI_FacA_S as String
DIM PC_SD_OI_FacB_N as Single
DIM PC_SD_OI_FacB_S as String
DIM PC_SD_OI_FacC_N as Single
DIM PC_SD_OI_FacC_S as String
DIM PC_SD_VL_N as SINGLE
DIM PC_SD_VL_S as STRING
DIM PC_SD_VL_FacA_N as Single
DIM PC_SD_VL_FacA_S as String
DIM PC_SD_VL_FacB_N as Single
DIM PC_SD_VL_FacB_S as String
DIM PC_SD_VL_FacC_N as Single
DIM PC_SD_VL_FacC_S as String
DIM FACTOR_A as Single
DIM FACTOR_B as Single
DIM FACTOR_C as Single
DIM A as integer
DIM B as integer
DIM C as integer
DIM A_S as STRING
DIM B_S as STRING
DIM C_S as STRING

A%=4
B%=7
C%=10
A_S$=STR$(A%)
B_S$=STR$(B%)
C_S$=STR$(C%)
FACTOR_A!=(2/(A%+1))
FACTOR_B!=(2/(B%+1))
FACTOR_C!=(2/(C%+1))


CUMPUT_OI_N# = 0
CUMCALL_OI_N# = 0
CUMPUT_VL_N# = 0
CUMCALL_VL_N# = 0

FOR i = FIRSTTICKERROW% TO LASTTICKERROW%
BID! = VAL(WorkingRawArray$(i, 11))
ASK! = VAL(WorkingRawArray$(i, 12))
OI! = VAL(WorkingRawArray$(i, 14))
CalcArray_OI#(i) = (OI! * (BID! + ASK!)) / 2
VOL! = VAL(WorkingRawArray$(i, 13))
CalcArray_VL#(i) = (VOL! * (BID! + ASK!)) / 2
IF WorkingRawArray$(i, 6) = "call" THEN
CUMCALL_OI_N# = CUMCALL_OI_N# + CalcArray_OI#(i)
CUMCALL_VL_N# = CUMCALL_VL_N# + CalcArray_VL#(i)
ELSE
CUMPUT_OI_N# = CUMPUT_OI_N# + CalcArray_OI#(i)
CUMPUT_VL_N# = CUMPUT_VL_N# + CalcArray_VL#(i)
END IF
NEXT i

TICKER$ = WorkingRawArray$(LASTTICKERROW%, 1)
CHARLEN% = LEN(TICKER$)
IF CHARLEN% > 4 THEN
CHAR5$ = MID$(TICKER$, 5, 1)
IF CHAR5$ = "/" THEN CHAR5$ = "-"
END IF
IF CHARLEN% > 3 THEN
CHAR4$ = MID$(TICKER$, 4, 1)
IF CHAR4$ = "/" THEN CHAR4$ = "-"
END IF
IF CHARLEN% > 2 THEN
CHAR3$ = MID$(TICKER$, 3, 1)
IF CHAR3$ = "/" THEN CHAR3$ = "-"
END IF
IF CHARLEN% > 1 THEN
CHAR2$ = MID$(TICKER$, 2, 1)
IF CHAR2$ = "/" THEN CHAR2$ = "-"
END IF
CHAR1$ = MID$(TICKER$, 1, 1)
TICKER$ = CHAR1$ + CHAR2$ + CHAR3$ + CHAR4$ + CHAR5$

PRINT TICKER$
FOR i=1 to FINALROWNUMBER_KILL%
If TICKER$=KILLArray(i) then EXIT SUB
NEXT i

DATARAW$ = MID$(WorkingRawArray$(LASTTICKERROW%, 8), 1, LEN(WorkingRawArray$(LASTTICKERROW%, 8)) - 11)
DATELEN% = LEN(DATARAW$)
SLASHCOUNT = 0
FOR j = 1 TO DATELEN% - 3
IF MID$(DATARAW$, j, 1) = "/" THEN
SLASHCOUNT% = SLASHCOUNT% + 1
SLASHARRAY%(SLASHCOUNT%) = j
END IF
NEXT j
MONTH_S$ = MID$(DATARAW$, 1, SLASHARRAY%(1) - 1)
DAY_S$ = MID$(DATARAW$, SLASHARRAY%(1) + 1, SLASHARRAY%(2) - SLASHARRAY%(1) - 1)
YEAR_S$ = MID$(DATARAW$, SLASHARRAY%(2) + 1, 4)
DAY_N% = VAL(DAY_S$)
MONTH_N% = VAL(MONTH_S$)
YEAR_N% = VAL(YEAR_S$)
DATADATE_N# = (YEAR_N%*10000) + (MONTH_N%*100) + DAY_N%
DATADATE_S$ = STR$(DATADATE_N#)
CSPRICE$ = WorkingRawArray$(LASTTICKERROW%, 2)
CSPRICENUM!=VAL(CSPRICE$)
CUMCALL_OI_S$ = STR$(CUMCALL_OI_N#)
CUMPUT_OI_S$ = STR$(CUMPUT_OI_N#)
PUTCALL_OI_N!=CUMPUT_OI_N#/CUMCALL_OI_N#
PUTCALL_OI_S$=STR$(PUTCALL_OI_N!)
CUMCALL_VL_S$ = STR$(CUMCALL_VL_N#)
CUMPUT_VL_S$ = STR$(CUMPUT_VL_N#)
PUTCALL_VL_N!=CUMPUT_VL_N#/CUMCALL_VL_N#
PUTCALL_VL_S$=STR$(PUTCALL_VL_N!)

FILENAMESAVE_OI$ = "E:\OptData\OI\"+TICKER$+"_oi.csv"
OPEN FILENAMESAVE_OI$ FOR APPEND AS #1
N = LOF(1)
CLOSE #1
IF N = 0 THEN
OPEN FILENAMESAVE_OI$ FOR APPEND AS #1
PRINT #3, "DATA,INDEX,CS,MKTCAP,MKTCAPMA,CUMCALLOI,CUMPUTOI,PutCall OI,PC SD OI,PC MA OI,MC MA2,MC MA3,INDIC MA2,INDIC MA3,DUMMY1,DUMMY2"
PRINT #3, "DATE,DATE,PRICE,999,"+ C_S$ +",CUMCALLOI,CUMPUTOI,1,2,"+C_S$+","+A_S$+","+B_S$+","+A_S$+","+B_S$+",DUMMY1,DUMMY2"
CLOSE #1
END IF

i=1
OPEN FILENAMESAVE_OI$ FOR INPUT AS #1
DO WHILE NOT EOF(1)
LINE INPUT #1, WorkingLineArray$(i)
IF RTRIM$(WorkingLineArray$(i)) = "" THEN EXIT DO
i=i+1
LOOP
CLOSE #1
FINALROWNUMBER% = i-1

OPEN FILENAMESAVE_OI$ FOR INPUT AS #1
FOR i = 1 TO FINALROWNUMBER%
FOR j = 1 TO 16
INPUT #1, WorkingFinalArray_OI$(i, j)
NEXT
NEXT
CLOSE #1

OPEN FILENAMESAVE_OI$ FOR APPEND AS #1
MKTCAP_N!=CSPRICENUM!*val(WorkingFinalArray_OI$(2,4))
MKTCAP_S$=STR$(MKTCAP_N!)
MKTCAP_FacA_N! = (MKTCAP_N!*FACTOR_A!)+(val(WorkingFinalArray_OI$(FINALROWNUMBER%,11))*(1-FACTOR_A!))
MKTCAP_FacA_S$ = STR$(MKTCAP_FacA_N!)
MKTCAP_FacB_N! = (MKTCAP_N!*FACTOR_B!)+(val(WorkingFinalArray_OI$(FINALROWNUMBER%,12))*(1-FACTOR_B!))
MKTCAP_FacB_S$ = STR$(MKTCAP_FacB_N!)
MKTCAP_FacC_N! = (MKTCAP_N!*FACTOR_C!)+(val(WorkingFinalArray_OI$(FINALROWNUMBER%,5))*(1-FACTOR_C!))
MKTCAP_FacC_S$ = STR$(MKTCAP_FacC_N!)
PC_SD_OI_N! = (PUTCALL_OI_N!-val(WorkingFinalArray_OI$(2,8)))/val(WorkingFinalArray_OI$(2,9))
PC_SD_OI_S$ = STR$(PC_SD_OI_N!)
PC_SD_OI_FacA_N! = (PC_SD_OI_N!*FACTOR_A!)+(val(WorkingFinalArray_OI$(FINALROWNUMBER%,13))*(1-FACTOR_A!))
PC_SD_OI_FacA_S$ = STR$(PC_SD_OI_FacA_N!)
PC_SD_OI_FacB_N! = (PC_SD_OI_N!*FACTOR_B!)+(val(WorkingFinalArray_OI$(FINALROWNUMBER%,14))*(1-FACTOR_B!))
PC_SD_OI_FacB_S$ = STR$(PC_SD_OI_FacB_N!)
PC_SD_OI_FacC_N! = (PC_SD_OI_N!*FACTOR_C!)+(val(WorkingFinalArray_OI$(FINALROWNUMBER%,10))*(1-FACTOR_C!))
PC_SD_OI_FacC_S$ = STR$(PC_SD_OI_FacC_N!)

IF FINALROWNUMBER%=2 Then
MKTCAP_FacA_N! = MKTCAP_N!
MKTCAP_FacA_S$ = STR$(MKTCAP_FacA_N!)
MKTCAP_FacB_N! = MKTCAP_N!
MKTCAP_FacB_S$ = STR$(MKTCAP_FacB_N!)
MKTCAP_FacC_N! = MKTCAP_N!
MKTCAP_FacC_S$ = STR$(MKTCAP_FacC_N!)
PC_SD_OI_FacA_N! = PUTCALL_OI_N!
PC_SD_OI_FacA_S$ = STR$(MKTCAP_FacA_N!)
PC_SD_OI_FacB_N! = PUTCALL_OI_N!
PC_SD_OI_FacB_S$ = STR$(MKTCAP_FacB_N!)
PC_SD_OI_FacC_N! = PUTCALL_OI_N!
PC_SD_OI_FacC_S$ = STR$(MKTCAP_FacC_N!)
End IF

PRINT #1, WorkingRawArray$(LASTTICKERROW%, 8)+","+DATADATE_S$+","+CSPRICE$+","+MKTCAP_S$+","+MKTCAP_FacC_S$+","+CUMCALL_OI_S$+","+CUMPUT_OI_S$+","+PUTCALL_OI_S$+","+PC_SD_OI_S$+","+PC_SD_OI_FacC_S$+","+MKTCAP_FacA_S$+","+MKTCAP_FacB_S$+","+PC_SD_OI_FacA_S$+","+PC_SD_OI_FacB_S$+",DUMMY1,DUMMY2"
CLOSE #1

FILENAMESAVE_VL$ = "E:\OptData\VL\"+TICKER$+"_VL.csv"
OPEN FILENAMESAVE_VL$ FOR APPEND AS #1
N = LOF(1)
CLOSE #1
IF N = 0 THEN
OPEN FILENAMESAVE_VL$ FOR APPEND AS #1
PRINT #3, "DATA,INDEX,CS,MKTCAP,MKTCAPMA,CUMCALLVOL,CUMPUTVOL,PutCall VOL,PC SD VOL,PC MA VOL,MC MA2,MC MA3,INDIC MA2,INDIC MA3,DUMMY1,DUMMY2"
PRINT #3, "DATE,DATE,PRICE,999,"+ C_S$ +",CUMCALLVOL,CUMPUTVOL,1,2,"+C_S$+","+A_S$+","+B_S$+","+A_S$+","+B_S$+",DUMMY1,DUMMY2"
CLOSE #1
END IF

i=1
OPEN FILENAMESAVE_VL$ FOR INPUT AS #1
DO WHILE NOT EOF(1)
i = i + 1
LINE INPUT #1, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #1
FINALROWNUMBER% = i-1
OPEN FILENAMESAVE_VL$ FOR INPUT AS #1
FOR i = 1 TO FINALROWNUMBER%
FOR j = 1 TO 16
INPUT #1, WorkingFinalArray_VL$(i, j)
NEXT
NEXT
CLOSE #1

OPEN FILENAMESAVE_VL$ FOR APPEND AS #1
PC_SD_VL_N! = (PUTCALL_VL_N!-val(WorkingFinalArray_VL$(2,8)))/val(WorkingFinalArray_VL$(2,9))
PC_SD_VL_S$ = STR$(PC_SD_VL_N!)
PC_SD_VL_FacA_N! = (PC_SD_VL_N!*FACTOR_A!)+(val(WorkingFinalArray_VL$(FINALROWNUMBER%,13))*(1-FACTOR_A!))
PC_SD_VL_FacA_S$ = STR$(PC_SD_VL_FacA_N!)
PC_SD_VL_FacB_N! = (PC_SD_VL_N!*FACTOR_B!)+(val(WorkingFinalArray_VL$(FINALROWNUMBER%,14))*(1-FACTOR_B!))
PC_SD_VL_FacB_S$ = STR$(PC_SD_VL_FacB_N!)
PC_SD_VL_FacC_N! = (PC_SD_VL_N!*FACTOR_C!)+(val(WorkingFinalArray_VL$(FINALROWNUMBER%,10))*(1-FACTOR_C!))
PC_SD_VL_FacC_S$ = STR$(PC_SD_VL_FacC_N!)

IF FINALROWNUMBER%=2 Then
PC_SD_VL_FacA_N! = PUTCALL_VL_N!
PC_SD_VL_FacA_S$ = STR$(MKTCAP_FacA_N!)
PC_SD_VL_FacB_N! = PUTCALL_VL_N!
PC_SD_VL_FacB_S$ = STR$(MKTCAP_FacB_N!)
PC_SD_VL_FacC_N! = PUTCALL_VL_N!
PC_SD_VL_FacC_S$ = STR$(MKTCAP_FacC_N!)
End IF

PRINT #1, WorkingRawArray$(LASTTICKERROW%, 8)+","+DATADATE_S$+","+CSPRICE$+","+MKTCAP_S$+","+MKTCAP_FacC_S$+","+CUMCALL_VL_S$+","+CUMPUT_VL_S$+","+PUTCALL_VL_S$+","+PC_SD_VL_S$+","+PC_SD_VL_FacC_S$+","+MKTCAP_FacA_S$+","+MKTCAP_FacB_S$+","+PC_SD_VL_FacA_S$+","+PC_SD_VL_FacB_S$+",DUMMY1,DUMMY2"
CLOSE #1

' UNCOMMENT BELOW WHEN RUNNING
PRINT #10, TICKER$+","+CSPRICE$

END SUB

------------------------------------------------------------------------------------------------------------------------------------------------------------

This is the version that does not compile. I had (most recent revision) set TKR_PREV(5000,2) and had all references to that variable set to TKR_PREV(i,1) or (i,j) and it compiled. Program bombed (but compiled) somewhere in the area where this code was called and I was starting to debug when I realized that my basic concept could be improved by changing data structure of underlying data.

Thnaks for looking at this.

dbish

MystikShadows
Veteran
Posts: 703
Joined: Sun Nov 14, 2004 7:36 am
Contact:

Post by MystikShadows » Thu May 05, 2005 10:37 am

Hi dbish....

You may want to try the command line compiler directly without going through FBIDE see what happens.

By the way, you might remember me from your other post. have you implemented that other part of my solution proposal? How did that go?
When God created light, so too was born, the first Shadow!

MystikShadows

Need hosting? http://www.jc-hosting.net

Interested in Text & ASCII development? Look no further!
http://www.ascii-world.com

dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

Post by dbish » Thu May 05, 2005 11:25 am

Hi there MystikShadows,

Yes, I do remember you and your previous suggestions have been very helpful indeed. As far as optimizing my routines - it is not really necessary. I am starting with 2 million files and distilling them into what I need. I am taking it one step at a time - write code, test, run it to accomplish the next step of the processing, and then moving on. Once I get code to run then I never need to re-run.

I am amazed how efficient QBASIC/FREEBASIC has been for this task. A piece of my processing might take 2-3 minutes run time (optimized it would be 60 seconds but I don't care) and I have made an update to 5000 end-result files. I usually work in Excel and just the thought of opening, manipulating, and closing 5000 files is mind boggling.

Regarding my problem at hand, I tried the suggestion of doing the compile from the command line. I am not sure the exact process but I tried from the command line:

c:\..path..\fbc.exe optdaily.bas

The program bombed shortly after it started with a generic Windows error message. I tried putting in a

PRINT "GOT TO HERE"
Sleep 1000

line to see where it bombs and it bombed very early (before my test message) so I am guessing it is still bombing on the compile.

Any other thoughts?

dbish

MystikShadows
Veteran
Posts: 703
Joined: Sun Nov 14, 2004 7:36 am
Contact:

Post by MystikShadows » Thu May 05, 2005 12:04 pm

I'm thinking that on the array WorkingRawArray that you have dimensioned as 1 to 250000, 1 to 19 maybe you can recode to loop through the same loop 19 times instead of attempting to acquire everything prior to processing would definitaly lighten the load...wouldn't really interfere with processing time (the possible extra time would probably be negligeable) and you'd save 18 pages of 250 Kb each.... as far as resource management goes. :-)
When God created light, so too was born, the first Shadow!

MystikShadows

Need hosting? http://www.jc-hosting.net

Interested in Text & ASCII development? Look no further!
http://www.ascii-world.com

dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

Post by dbish » Thu May 05, 2005 12:28 pm

I looked closer and changed the DIM parameters on my big arrays - I probably don't need them that big. I don't see that this is a memory resource error. Why would it compile when the TKR_PREV array was (5000,2) and not at (5000) one dimensional? This feels like something else.

When I get a chance I could try your suggestion but it will require changing my logic further down (call as needed versus assuming it is all there).

My next approach is to split this whole program into two pieces.

SETUP of PROGRAM (DIMS etc
.... PART A Create/manipulate TKR_PREV. This peice could be placed after Part B finishes
..............PART B read large arrays, extract data, update FILENAMESAVE_OI and VL files
..............PART B capture data for TKR_NEW
.... PART A Process TKR_NEW and compare to TKR_PREV

Part B is the large array reads and multiple calls to the subroutine to provide daily updates to my end-result files (called by looping FILENAMESAVE_OI and FILENAMESAVE_VL). This piece is independent of rest (Part A) which is trying to look at activity of what was updated, compare to previous day's activity, and generate an exception report. While the two activities are independent I plan to run this code daily with a scheduler and need part B to be finished before part A starts. Perhaps there are ways around that (can Part B call part A as a final command?)

I know the above is confusing but that is how I developed it. Hopefully it is somewhat understandable.

I am still stumped by making the TKR_PREV array simpler and smaller should cause a compile error. I did a lot of testing (commenting out various lines) to see what causes the problem and it clearly is that statement change.

Stumped!

dbish

User avatar
Kyle
Veteran
Posts: 107
Joined: Thu Apr 14, 2005 2:41 pm

Post by Kyle » Thu May 05, 2005 12:35 pm

What does your code actually do?

dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

Post by dbish » Thu May 05, 2005 12:58 pm

My project has evolved to the following:

First, I have distilled 2 million stock market related data files into 2400 files of the name form TICKER_OI.csv and TICKER_VL.csv. I download a daily data update file which is 140000 rows and has new data to update the 4800 TICKER_xx.csv files on my computer. This daily update file has 2-100 rows for each ticker.

The code (mentioned in prior post) does some housekeeping in the beginning and works with a TKR_PREV file which is really not needed until the last step (and should be moved there). The section starting with line 93 (OPEN FILENAMECURRENT$ FOR INPUT AS #1) is the part that reads the daily file, puts it into a working array for my subroutine TICKERDO which figures out how many rows belong with each ticker and updates the relevant files. It also captures some data and writes it to TKR_NEW.csv.

After all my ticker files have been updated the code (starting with line 129 - OPEN "E:\OptData\SUMMARY_"+DATESTRING$+".csv" FOR APPEND AS #3) opens TKR_PREV and TKR_NEW and reads the files into arrays which then try to figure out if some tickers were not updated, some new tickers appeared, and if significant price changes happened. I have not finished debugging this last piece yet. The part with reading the daily update and updating the data files works (I think).

Hopefully this is understandable.

dbish

dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

Post by dbish » Thu May 05, 2005 2:11 pm

I have hived off the Exception report processing into a separate program which should be small and easy. I am still getting the same "won't compile error".

New, smaller code as follows:

----------------------------------------------------------------------------------------------------------------------------------
OPTION BASE 1
DEFINT A-Z
DIM OPTIONLINE AS STRING
DIM TKR_PREV(5000) as STRING
DIM TKR_NEW(5000,4) as STRING
DIM FINALROWNUMBER_NEW AS INTEGER
DIM FINALROWNUMBER_PREV AS INTEGER
DIM TKR_NEW_START as INTEGER
DIM TKR_PREV_START as INTEGER
DIM CHAR1 AS STRING
DIM TKR_PREV_SHORT as STRING
DIM TKR_PREVLEN as Integer

SHELL "dir E:\Optdata\OI\*.csv /b > c:\temp2\TKR_PREV.csv"
i = 0
OPEN "C:\Temp2\TKR_PREV.csv" FOR INPUT AS #5
DO WHILE NOT EOF(5)
i = i + 1
LINE INPUT #5, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #5
FINALROWNUMBER_PREV% = i
OPEN "C:\Temp2\TKR_PREV.csv" FOR INPUT AS #5
FOR i = 1 TO FINALROWNUMBER_PREV%
INPUT #5, TKR_PREV$(i)
NEXT i
CLOSE #5
For i = 1 to FINALROWNUMBER_PREV%
TKR_PREVLEN%=LEN(TKR_PREV$(i))-7
TKR_PREV_SHORT$=LEFT$(TKR_PREV$(i),TKR_PREVLEN%)
TKR_PREV$(i)= TKR_PREV_SHORT$
Next i
KILL "c:\temp2\TKR_PREV.csv"
OPEN "c:\temp2\TKR_PREV.csv" FOR APPEND as #5
For i = 1 to FINALROWNUMBER_PREV%
PRINT #5, TKR_PREV$(i)
Next i
CLOSE #5


OPEN "E:\OptData\SUMMARY.csv" FOR APPEND AS #3

OPEN "c:\temp2\TKR.csv" FOR INPUT AS #2
i=0
DO WHILE NOT EOF(1)
i = i + 1
LINE INPUT #1, OPTIONLINE$
CHAR1$ = LEFT$(OPTIONLINE$, 1)
IF CHAR1$ = "" THEN EXIT DO
LOOP
CLOSE #2
FINALROWNUMBER_NEW% = i
OPEN "c:\temp2\TKR.csv" FOR INPUT AS #2
FOR i = 1 TO FINALROWNUMBER_NEW%
FOR j = 1 TO 4
INPUT #2, TKR_NEW(i, j)
NEXT
NEXT
CLOSE #2
If FINALROWNUMBER_PREV% >= FINALROWNUMBER_NEW% then
TICKER_NEW_START%=0
Else
TICKER_NEW_START%=1
END IF

IF TICKER_NEW_START=0 THEN
FOR i=1 to FINALROWNUMBER_PREV%
For j= 1 to FINALROWNUMBER_NEW%
If TKR_PREV$(i)=TKR_NEW(j,1) then
If ((val(TKR_NEW$(i, 2))/val(TKR_PREV$(j,2))-1))<-.15 then
PRINT #3,"SPLIT?,"+TKR_NEW(i, 1)+","+TKR_NEW$(i, 2)+","+TKR_PREV$(j,2)
GOTO NEXTI_P1
END IF
END IF
NEXT j
PRINT #3,"MISSING?"+TKR_PREV$(i)+",--,--"
NEXTI_P1:
NEXT i
FOR i=1 to FINALROWNUMBER_NEW%
For j= 1 to FINALROWNUMBER_PREV%
If TKR_NEW(i, 1)=TKR_PREV(j,1) then
GOTO NEXTI_P2
END IF
NEXT j
PRINT #3,"NEW?,"+TKR_NEW(i, 1)+",--,--"
NEXTI_P2:
NEXT i
ELSE
FOR i=1 to FINALROWNUMBER_NEW%
For j= 1 to FINALROWNUMBER_PREV%
If TKR_NEW(i, 1)=TKR_PREV(j,1) then
If ((val(TKR_NEW$(i, 2))/val(TKR_PREV$(j,2))-1))<-.15 then
PRINT #3,"SPLIT?,"+TKR_NEW(i, 1)+","+TKR_NEW$(i, 2)+","+TKR_PREV$(j,2)
GOTO NEXTI_N1
END IF
END IF
NEXT j
PRINT #3,"NEW?"+TKR_NEW(i, 1)+",--,--"
NEXTI_N1:
NEXT i
FOR i=1 to FINALROWNUMBER_PREV%
For j= 1 to FINALROWNUMBER_NEW%
If TKR_PREV$(i)=TKR_NEW(j,1) then
GOTO NEXTI_N2
END IF
NEXT j
PRINT #3,"MISSING?,"+TKR_PREV$(i)+",--,--"
NEXTI_N2:
NEXT i
END IF

END
----------------------------------------------------------------------------------------------------------------------------------
I have not completely debugged this (since it won't compile). However, memory errors should be non-existent. Same error - the DIM TKR_PREV as a one dimensional array is giving it fits.

Thoughts?

dbish

dbish
Coder
Posts: 19
Joined: Tue Apr 12, 2005 7:37 pm

Post by dbish » Fri May 06, 2005 1:02 am

HOORAY - I CAN CALL EVERYBODY OFF.

No, I don't know what the problem was but I stripped the code down to the smallest initial part I could and ran it. Added the next few lines, etc. Found that FB didn't like something around the ((val(TKR_NEW$(i, 3))/val(TKR_NEW$(i,4))))<.8 lines (this is slight change from code above).

Somehow - it now compiles again and I can get to debugging.

Thanks to everyone who helped out here.

dbish

Post Reply