|
JCapper Message Board JCapper 101
|
| Home |
Register
Log In |
| By | Calc Races - Datbase Locked -Pop Up | |
| msammons 8/22/2012 1:53:36 PM | When running calculate races (HDW data), I get up to 20 popups saying: "Database currently locked. Attempt auto recovery? Reset DB Connection and continue on? (Recommended)" Is there some way of disabling this popup? I am trying to avoid clicking "OK" 20 times over 5 minutes or so, while the days races are caluclated.
| |
| jeff 8/22/2012 1:58:56 PM | 20 or more times? Yikes! Before diving in, I am going to say that what you are describing is not normal behavior. In the Laptop Reviews thread (click here) I described some of my "older" machines. On my Toshiba laptop (mfg 2007) I get the occasional database locked message - maybe 3 or 4 of them per day. Keep in mind that I probably get current scratches and run a calc races 50 or more times each and every race day. That's (normally) with the following program windows open: JCapper Main Module, JCapper Report Viewer, JCapper SQL Mode Reports Module, JCapper WagerHistory Module, Firefox with the following tabs: Rebate House ADW in 1st tab, Twinspires so that I can watch track video in the 2nd tab, email client open in the 3rd tab, JCapper Message Board or Paceadvantage open in the 4th tab. On the now defunct desktop pc (new in 2003) I might get 5-8 database locked messages over the course of a race day. On my new laptop, I have yet to get even a single database locked message box. (I'm not kidding about that.) Can the message box simply be disabled? No. The message box can not be disabled. It is there for a reason. Actually, it is there for two reasons: 1. It gives Windows a chance to catch up and clear the .ldb lock file that it places on .mdb files whenever they are accessed by the database driver. 2. It gives you a recovery path so that the Calc Races can run to completion. Without the message box, you would get an error message instead - and have to rerun the Calc Races in its entirety - and hope that you didn't run into another database locked situation. Why does the database locked condition occur? From what I can see on the machines I am using, and from what I have been able to determine by reading about how Windows handles multiple apps running at the same time: Whenever a machine runs low on memory it halts execution of apps running in open program windows. (This includes apps running in the background not launched by the user - or even without the user being aware they are running.) Once everything has been halted, Windows then writes everything in memory to the swap or paging file, clears everything in memory, and then reloads everything in the swap or paging file back into memory - and only then - begins running apps again. This process can take several minutes. And depending on what is running in the background and how much memory the apps running are (collectively) using can sometimes cause a machine to wind up in a state where the machine is constantly halting program execution so that the machine can continually go through the exercise of writing to and reading from the swap file. Based that you posted "20 or more messages per calc races" I am going to make an educated guess that you are experiencing something very similar to what I just described. What Can I Do? In my opinion, solving it becomes a question of getting at least some of the following questions answered: "Is my machine going into swap file hell and if so, why?" Q. How many race card files are loaded into the program so that they are handled by a single calc races routine? Q. Put another way: How many race card files are being handled when you click the Calc Races button? Q. How many program windows (that you are aware of) are open and running while you are running a calc races? Q. Are there any open program windows that you can close down before running a calc races? Q. How big is the c:\JCapper\Exe\JCapper2.mdb file? (If you right click this file and select properties, what is the file size?) Get me answers to the above questions and I'll try to help. Again, what you are describing is not normal behavior. -jp .
| |
| msammons 8/22/2012 5:31:36 PM | Thank you for the very detailed reply. My PC is not that bad - dual core AMD 2200 Mhz cpu, with 4 GB ram. I reran the calculate races with nothing else running and got 8 messages (on heavy days like Saturdays it can be 20). I assume the problem is that I run a calculate races on all HDW races (I mean I run all of todays races at the same time). I would not think it would matter but I never get scratches before running all the days races. The c:\JCapper\Exe\JCapper2.mdb file is 1 GB. I think I can write a simple macro which will click "yes" (actually hits "enter") everytime the popup comes up - I just wanted to make sure there was not another option I was overlooking.
| |
| jeff 8/22/2012 7:11:33 PM | First thing to try... Organizing the Race Day There are 18 thoroughbred tracks running today (Wed August 22, 2012.) Instead of loading all 18 card files onto a single folder try breaking the race day up into 3 groups of 6 files each. The Card Loader in the DFM was designed explicitly for this purpose. Below I've posted screenshots which will (hopefully) serve to illustrate how to make it all work. Screenshot #1: http://www.jcapper.com/messageboard/avatars/dfmcardloader_activedatafolders01.jpg The above screenshot shows my DFM (Data Folder Manager.) Note that I have set up 3 separate data folders. The first folder (the one that is checked) is the current active data folder. I can change to another simply by clicking the box for that folder and hitting the Save button. Even though I CAN do that, there is seldom a reason (none unless I want to do scratches manually for a gate scratch) for doing that on race day. The important thing for making this work is that you have to create at least 2 additional folders and point to them in the DFM first. If you don't do that, it's easy to have the DFM pointed at the same paths for folder1, folder2, and folder3 (which defeats the purpose of breaking the day up into 3 separate groups of files.) Also note that if you do change the current active data folder during a race day - you have to pull up the DFM and change it back before downloading files for the next race day - otherwise you will end up saving tomorrow's files to the wrong folder. (So get in the habit of checking the current active data folder in the DFM before using the HDW File Downlodd Tool to queue up files and download them.) The current folder structure on my machine looks like this: c:\2012 This is my yearly folder for calendar year 2012. c:\2012\Q1_2012 c:\2012\Q2_2012 c:\2012\Q3_2012 c:\2012\Q4_2012 These are subfolder that I created beneath my 2012 calendar year folder. They are also my quarterly calendar folders that are used as active data folders (at the appropriate time) during calendar year 2012. These are the folders where my HDW files are saved to. These are also the folders where my .jcp and .xrd files are built - and they are also the folders where I point to when I run build database routines. c:\2012\Folder2 c:\2012\Folder3 These are sub folders that I created on the yearly folder for calendar year 2012. The sole purpose of these folders is to facilitate breaking up race days into smaller groups of 3 as opposed to a single larger group of files. Fewer files handled during a Calc Races translates to less RAM used during a Calc Races - which has the ability to cut down on the incidence rate of the database locked condition. Screenshot #2: http://www.jcapper.com/messageboard/avatars/dfmcardloader_byposttime01.jpg The above screenshot shows the DFM Card Loader with all 18 of today's .jcp files dragged and dropped onto the post time list - where the interface automatically sorts them according to post time for R1. From here I can drag and drop the first group of 6 from the post time list onto the pane for Folder1, the second group of 6 from the post time list onto the pane for Folder2, and the third group of 6 from the post time list onto the pane for Folder 3. Screenshot #3: http://www.jcapper.com/messageboard/avatars/dfmcardloader_byposttime02.jpg The above screenshot shows the DFM Card Loader with all 18 of today's .jcp files dragged and dropped onto the panes for Folder1, Folder2, and Folder3. From here, I can load them into the program by clicking the Select ALL and Load button. Screenshot #4: http://www.jcapper.com/messageboard/avatars/enhancedsettings_08222012.jpg This screenshot shows my Enhanced Settings Module - where I've ever so crudely highlighted the setting I want to draw your attention to using a yellow mouse pen. You have the option for making the Calc Races button on the Main Module and the XML button on Scratch BOT render either as a single button - or as 3 separate buttons that highlight as you hover your mouse cursor over them. Here, I've persisted the setting that makes them render as 3 buttons. (You can see what that looks like in the next screenshot.) Screenshot #5: http://www.jcapper.com/messageboard/avatars/calcraces3buttons.jpg The above screenshot shows the 3 button Calc Races interface on the JCapper Main Module. Here, I am about to run a Calc Races on Folder 1. Note that because I used the DFM Card Loader to load 6 card files onto Folder 1, when I click this button, the Calc Races routine that gets run is for those 6 card files only. The advantages for doing this are: • Fewer card files handled by a Calc Races routine translates to less RAM used during a Calc Races routine. One side effect to this is that you should be less likely to get a database locked message. A second advantage to this is that a Calc Races for one third of the day takes up a lot less time than a Calc Races for the entire day. (If I have organized the race day properly, I can run a Calc Races for the files on Folder 1 - and I have a few hours before I HAVE to run a Calc Races for the files on Fodlers 2 - and (eventually) Folder 3. • Notice that before loading today's card files into the program, I organized them according to post time for R1. This means that when I run a Calc Races for Folder 1, there is no (precious) time wasted by running a Calc Races for race cards that haven't even put up their scratches yet - In just about all cases (provided I have organized the race day properly) races for the files on Folders 2 and 3 won't even start for another few hours. At any point in the race day, I can get scratches for the cards on Folder 2, and run a Calc Races for the files loaded into Folder 2. My normal way of doing things has me doing that separately about 60 mtp for the first card on that folder - and then repeating at regular intervals throughout the race day. My normal way of doing things has me doing the same thing for Folder 3. Try it. You may find that by using less RAM with each Calc Races that the incidence rate for the database locked condition goes way down. More to come... -jp . ~Edited by: jeff on: 8/22/2012 at: 7:11:33 PM~
| |
| jeff 8/22/2012 7:13:36 PM | I have edited the above post a few times since posting the original. If the timestamp for it isn't 8/22/2012 7:11:33 PM then REFRESH your browser window to see the most current version of what I am trying to get across. -jp .
| |
| msammons 8/22/2012 8:56:47 PM | Good idea on breaking the days races into 3 time zones. I am working on implementing that right now. I appreciate the extensive discussion and instructions!
| |
| jeff 8/23/2012 10:56:25 AM | You wrote: "The c:\JCapper\Exe\JCapper2.mdb file is 1 GB." My reply: What I am about to post next may or may not impact performance and RAM usage during a Calc Races on your particular machine. (It "should" impact performance, but each chipset configuration comes with its own idiosyncrasies.) 1.0 gigabytes is 1000 megabytes. That's roughly 350 times the file size of the empty JCapper2.mdb file that comes with the program when you download it. Any time that the database driver is called upon to read from or write to the tables in an .mdb file, it has to load the .mdb file into memory - using up RAM in the process. Loading a 1000 megabyte .mdb file into memory "should" use more RAM (and have a greater impact on the machine's performance) than if the database driver were called upon to load a 3 megabyte .mdb file into memory to read the same info. This train of thought has led me to consider something that had not occurred to me previously. It might not be a bad idea for me as the program's author to stress test the Calc Races process and see just how much impact (on a variety of machines) running the Calc Races has when the program is connected to a giant JCapper2.mdb file vs. running a Calc Races while connected to a tiny file - for example the fresh blank (empty) copy sitting in the JCapperBuild folder... And if it turns out there is a significant difference on "most" machines - The next logical step would be to introduce a third .mdb file - one that gets touched ONLY during a Calc Races - and nowhere else in the program. If you are still getting database locked messages after breaking up the race day into 3 groups of files instead of a single (bigger) group of card files - Try this: 1. Rename the JCapper2.mdb file in the c:\JCapper\Exe folder. from: c:\JCapper\Exe\JCapper2.mdb --to: c:\JCapper\Exe\JCapper2_SAVED_TodaysDate.mdb 2. Copy the fresh blank copy of the JCapper2.mdb file created in the JCapperBuild folder by the Extractor/Installer from your last program install to the c:\JCapper\Exe folder. from: c:\JCapperBuild\JCapper2.mdb --to: c:\JCapper\Exe\JCapper2.mdb 3. If you are operating in sql mode, use the JCapper2 Imports Screen in the Utilities area of the WagerHistory Module to import the following tables: • SQL Mode F-Factor Setup • SQL Mode Customizable HTML Report Layout Source file: c:\JCapper\Exe\JCapper2_SAVED_TodaysDate.mdb Target file: c:\JCapper\Exe\JCapper2.mdb And then TEST performance on your machine by running a Calc Races. Let me know if it makes a difference on your machine. -jp .
| |
| jeff 9/1/2012 3:31:35 PM | Special Download Package (Main Module Only) August 31, 2012 Main Module – Improved Handling of Calc Races Database Locked Condition Background Info - Why does the database locked condition occur? From what I can see on the machines I am using, and from what I have been able to determine by reading about how Windows handles multiple apps running at the same time: Whenever a machine runs low on memory it halts execution of apps running in open program windows. (This includes apps running in the background not launched by the user - or even without the user being aware they are running.) Once everything has been halted, Windows then writes everything in memory to the swap or paging file, clears everything in memory, and then reloads everything in the swap or paging file back into memory - and only then - begins running apps again. This process can take several minutes. And depending on what is running in the background and how much memory the apps running are (collectively) using can sometimes cause a machine to wind up in a state where the machine is constantly halting program execution so that the machine can continually go through the exercise of writing to and reading from the swap file. During a Calc Races routine, logic in the Calc Races routine itself performs hundreds of read/write operations to the tables in the c:\JCapper\Exe\JCapper2.mdb file. Windows has a mechanism for securing .mdb database files. When a program is actively engaged in the act of reading data from or writing data to an .mdb database file, Windows “locks” the file so that other apps (and/or users) can’t write to that file while it’s in use. The mechanism for doing this is to put a “lock” file on the same folder as the .mdb file. Lock files have the same file name as the .mdb file except for the file extension. The file extension for a lock file are the characters “.ldb” (without the quotes.) If you were to sit and watch the c:\JCapper\Exe folder during a SQL Calc Races, you would see a file named JCapper2.ldb (that’s the lock file) placed on the folder whenever the Main Module is reading from or writing to the JCapper2.mdb file. You would also see the JCapper2.ldb lock file cleared (deleted) from the folder each time the Main Module isn’t actively engaged in reading data from or writing to the JCapper2.mdb file. The database locked condition comes into play when a machine runs out of memory during a Calc Races and has to start writing to the swap file. Under those circumstances, Windows is slow to clear the lock file. When the JCapper Main Module attempts to read from or write to its own database file – but can’t because Windows hasn’t been able to clear the lock file from the folder – a database locked error message is the result. Database Locked Handling Description (Old Way) Up until now, the way that I have handled a database locked error inside of the JCapper Main Module was to present the user with a message box displaying a database locked message while at the same time offering the user an “auto recovery” option that when clicked allows the Calc Races routine to be run to completion. What you may not have known is what goes on behind the scenes (on your machine) when that database locked message appears. I created that database locked message box in such a way that the Calc Races routine itself has to stop executing while the message is displayed. This gives Windows the time it needs to “catch up” and clear the lock file. By the time you click the message and select the auto recovery path, the lock file has long been deleted from the folder – enabling the .mdb file to be read from and written to again. That strategy might be clumsy and inelegant. But it works. Improved Handling of Calc Races Database Locked Condition The Main Module in this program update comes with a new (better) way of handling the database locked condition. Database Locked Handling Description (New Way) During a Calc Races routine, the JCapper Main Module in this program update handles the presence of an .ldb lock file on the JCapper2.mdb file in the following manner: 1. The Calc Races routine is halted for a few seconds. The folder where the Calc Races is being run is tested for the presence of the lock file. In most cases (on most machines) halting the routine for a few seconds will give Windows the chance to “catch up” and clear the lock file. If testing reveals that the lock has been cleared, program execution of the Calc Races picks up where it left off – without the user ever being made aware (via a “database locked” message) that a database locked condition was encountered. 2. If testing reveals that the lock file is still there, the Calc Races routine is halted for another 2-3 seconds – giving Windows a second chance to “catch up.” The folder where the Calc Races is being run is once again tested for the presence of a lock file. In almost all cases (on most machines) halting the routine a second time for a few seconds gives Windows the time it needs to “catch up” and clear the lock file. The folder is retested, and if testing reveals that the lock has (finally) been cleared, program execution of the Calc Races picks up where it left off – again without the user ever being made aware (with a “database locked” message) that a database locked condition was encountered. 3. However, if at this point testing reveals that the lock file is still there, the assumption is made that Windows needs more than just a second or two – and a “database locked” message is generated – and you will have to click through it and select the auto recovery path to restart the Calc Races so that it can be run all the way to completion. Install Instructions: 1. Close down all open JCapper program windows, log into the JCapper message board and go to the program downloads page. 2. Save the Main Module Only Special Download Package (published August 31, 2012) - filename: MainModuleOnly08312012.exe to your hard drive. (Hint: c:\JCapperBuild is a good place to save it.) 3. Double click the download package file (filename: MainModuleOnly08312012.exe) to run the extractor. The extractor will copy the .exe file for the new Main Module (and only the Main Module) to your c:\JCapper\Exe folder, overwriting the existing .exe file for the Main Module, and then on most machines, will launch the Main Module. That’s It! There’s nothing else to install – and no import routines to be run. After performing the above install you should be able to use the new Main Module to run Calc Races routines in both SQL and PlayList File Modes – and you should find that the number of database locked messages encountered goes down noticeably. Enjoy, -jp .
| |
| Charlie James 9/2/2012 12:05:02 PM | Did the main module only install yesterday. Number of database locked messages since then? Nada. Muchas Gracias Muchacho
|
| Reply |
|
Copyright © 2018 JCapper Software back to the JCapper Message Board www.JCapper.com |