Database Handicapping Software- JCapper

JCapper Message Board

          General Discussion
                      -- Data Folder Structure for the coming New Year

Home Register
Log In
By Data Folder Structure for the coming New Year
jeff
1/1/2021
10:29:56 AM		It's that time again. (Time to create folder structure for the coming new year.)

One of the things you NEVER want to do is mix and match files from different years on the same folder. The file names of data and results files that you download do not contain a year element.

If you fail to create (and use) folder structure for the new year:

Not only do you run the risk of overwriting 2020 files with 2021 files... But you also risk corrupting your 2020 databases in the event you run a db build using a 2020 data file and a 2021 results file (or vice versa) for file names involving the same track code, month, and day of month.

I (personally) use a single folder for each year with 4 separate sub folders (one per calendar quarter) and 2 additional sub folders (to facilitate breaking up my race day into multiple groups in the DFM Card Loader.)


My old 2020 folder structure looks like this:

C:\2020
C:\2020\Folder2
C:\2020\Folder3
C:\2020\Q1_2020
C:\2020\Q2_2020
C:\2020\Q3_2020
C:\2020\Q4_2020


My new 2021 folder structure looks like this:

C:\2021
C:\2021\Folder2
C:\2021\Folder3
C:\2021\Q1_2021
C:\2021\Q2_2021
C:\2021\Q3_2021
C:\2021\Q4_2021


As new files for 2021 become available for download - and for the first several days of the new year - I will be toggling back and forth between folders inside of the DFM - being EXTREMELY careful that all 2020 files are downloaded to my C:\2020\Q4_2020 folder - and that all 2021 files are downloaded to my C:\2021\Q1_2021 folder.

Mode 4 is the way to go the first time you run a build database on your new 2021 folder.

Use Mode 5 for each subsequent build on that folder thereafter.



The Easy Way to Create Folder Structure for each New Calendar Year:

Beginning with all versions of JCapper published 12-31-2017 and later:

The Data Folder Mgr contains a MENU item labeled “Create Data Folders for New Year” (without the quotes.)

Launch the DFM, click MENU, and select “Create Data Folders for New Year” (without the quotes.)

This will launch the Year End Folder Creation Tool.

From here, click the Create Data Folders for New Year button – then key in the 4 digit year for the new folder(s) you wish to create (for example "2021" without the quotes) and answer Yes at the confirmation prompt.

The interface will create a new set of folders for the 4 digit year you keyed in – and display a detailed confirmation message at the end.


Happy New Year Everyone,

-jp

.



Reply
georgekos
10/6/2021
9:49:06 AM		The New Year is coming...so a Question?

I want to have a Continuous 12 Month History for Analysis Purpose...

What is BEST WAY to do this?
Every Month Delete Oldest Month and add New Month to my Folder?

HELP will be appreciated...

GeorgeKos
at Poolside in Florida



Reply
jeff
10/8/2021
1:16:15 PM		Maintaining a continuous n months of history data is completely separate from the folder structure you decide to use for storing data and results files.

Maintaining a continuous n months of history data and having it available on demand for research purposes involves running regular DBBuild routines and making regular backups.

Below is what you need to know for both Playlist File Mode and SQL Mode.



If you are operating JCapper in PlayList File Mode:

I recommend using the Exports, Quick Index Create New File and Exports, Quick Index Append to Existing File options from the Data Window MENU.

Link to a thread from 2010 where I posted instructions for creating Quick Index File(s) spanning multiple folders: here.

Link to a thread from 2015 where I posted instructions and screenshots starting with the 6th post down from the top:
here.

If you are operating JCapper in Playlist File Mode:

You should always be able to use the Data Window MENU to create Quick Index .txt file(s) spanning multiple folders containing the desired n months of data you want for research purposes.

You're not likely to ever run into a Windows file size limitation for the PL_Profile.txt file where your data is stored if you are strictly operating in Playlist File Mode.

That said, rebuilding a quarterly data folder from scratch does take some time.

Therefore, it's a good idea to get into the habit of making regular backups of the PL_Profile.txt file. (Better to have a good recovery point in the event of some unforeseen emergency than having to rebuild a quarterly folder from Scratch in Mode 1.





If you are operating JCapper in SQL Mode:

  1. I recommend using Mode 3 for the very first DBBuild on the very first folder being built whenever you are using a new c:\JCapper\Exe\JCapper2.mdb file.


  2. Use Mode 4 for the very first DBBuild on the first calendar day on for a new folder whenever you are using an existing c:\JCapper\Exe\JCapper2.mdb file.


  3. Use Mode 5 after that for subsequent daily DBBuilds on the current folder until you reach the very last calendar day for that quarter whenever you are using an existing c:\JCapper\Exe\JCapper2.mdb file..


  4. On the very first day of the next new calendar quarter change the current active data folder in the DFM to the appropriate new folder.

    Then begin using that folder for the data and result files of that new quarter.

    And jump back to step 2 (above) for your DBBuild routines.


  5. I STRONGLY recommend making regular backup copies of your c:\JCapper\Exe\JCapper2.mdb file.

    The more recent your last known good backup the easier it is to recover in the event of an unforeseen emergency.


  6. Always be aware that the Microsoft Windows upper file size limit for Access .mdb files is 2.0 gigabytes.


  7. Always be aware you risk breaking the 2.0 gigabyte file size limit depending on the Mode you are using for your DBBuild routines.


  8. Using Mode 5 daily DBBuilds (as described above) I can generally squeeze about 3 years into the StarterHistory Table inside of a JCapper2.mdb file.

    But once a JCapper2.mdb file reaches about 1.65 gigabytes in file size:

    It's time to archive the current JCapper2.mdb file and replace it with a fresh blank copy of the JCapper2.mdb file pulled from the c:\JCapperBuild folder.


  9. Using Mode 4 Monthly or Quarterly DBBuilds (as described above) I can generally squeeze about 2 years into the StarterHistory Table inside of a JCapper2.mdb file.


    10. But once a JCapper2.mdb file reaches about 1.4 gigabytes in file size:

    It's time to archive the current JCapper2.mdb file and replace it with a fresh blank copy of the JCapper2.mdb file pulled from the c:\JCapperBuild folder.



If you are operating JCapper in SQL Mode in accordance with the above recommended steps:

You should always have a continuous n months of data available for research purposes (for any time period) sitting in the StarterHistory Table inside of:

Your current JCapper2.mdb file.

One of your many backup/archived copies of the JCapper2.mdb file.



-jp

.



~Originally Posted by: jeff  on:  10/7/2021  at:  12:43:39 PM~

~Edited by: jeff  on:  10/8/2021  at:  1:16:15 PM~

Reply
georgekos
12/21/2021
11:12:58 AM		Jeff: Can I not just keep a STARTER HISTORY Table with 12 months?

and just delete Oldest Month every Month?

so that would give me a contiguous 12 Month always?

Thank you... GeorgeKos

Reply
jeff
12/25/2021
10:55:48 AM		Sure. You can do it that way if you want.

Just understand going in that you will be in a constant state of deleting/losing older data from your StarterHistory table.

If you want to operate that way I suggest the following:

  1. Use the above recommended folder structure for each new year.

    That way you won't be mixing data files from different years on the same folder. And you won't be losing older data files.


  2. Make regular backups of your c:\2004JCapper.mdb and c:\JCapper\Exe\JCappe2.mdb filles.

    That way you'll always have a good recovery point should the need arise.



-jp

.



Reply
jeff
12/31/2021
1:33:07 PM		Data Folder Structure for 2022:

It's that time again. (Time to create folder structure for the coming new year.)

One of the things you NEVER want to do is mix and match files from different years on the same folder. The file names of data and results files that you download do not contain a year element.

If you fail to create (and use) folder structure for the new year:

Not only do you run the risk of overwriting 2021 files with 2022 files... But you also risk corrupting your 2021 databases in the event you run a db build using a 2021 data file and a 2022 results file (or vice versa) for file names involving the same track code, month, and day of month.

I (personally) use a single folder for each year with 4 separate sub folders (one per calendar quarter) and 2 additional sub folders (to facilitate breaking up my race day into multiple groups in the DFM Card Loader.)


My old 2021 folder structure looks like this:

C:\2021
C:\2021\Folder2
C:\2021\Folder3
C:\2021\Q1_2021
C:\2021\Q2_2021
C:\2021\Q3_2021
C:\2021\Q4_2021


My new 2022 folder structure looks like this:

C:\2022
C:\2022\Folder2
C:\2022\Folder3
C:\2022\Q1_2022
C:\2022\Q2_2022
C:\2022\Q3_2022
C:\2022\Q4_2022


As new files for 2022 become available for download - and for the first several days of the new year - I will be toggling back and forth between folders inside of the DFM - being EXTREMELY careful that all 2021 files are downloaded to my C:\2021\Q4_2021 folder - and that all 2022 files are downloaded to my C:\2022\Q1_2022 folder.

Mode 4 is the way to go the first time you run a build database on your new 2022 folder.

Use Mode 5 for each subsequent build on that folder thereafter.



The Easy Way to Create Folder Structure for each New Calendar Year:

Beginning with all versions of JCapper published 12-31-2017 and later:

The Data Folder Mgr contains a MENU item labeled “Create Data Folders for New Year” (without the quotes.)

Launch the DFM, click MENU, and select “Create Data Folders for New Year” (without the quotes.)

This will launch the Year End Folder Creation Tool.

From here, click the Create Data Folders for New Year button – then key in the 4 digit year for the new folder(s) you wish to create (for example "2022" without the quotes) and answer Yes at the confirmation prompt.

The interface will create a new set of folders for the 4 digit year you keyed in – and display a detailed confirmation message at the end.



Scratch BOT

Newer Build 198 versions of Scratch BOT enable you to define a folder where your ScratchesAndChanges .xml files are automatically archived.

Basic Operating Instructions:

1. Manually create a new folder (use any drive or path on your system) where you want Scratch BOT to automatically archive the ScratchesAndChanges .xml files that you will download from Equibase during calendar year 2022 when you click the XML or Bot Mode buttons.

Note: Starting with calendar year 2018, I created folder structure beneath my c:\JCapper\ folder that looks like this:

c:\JCapper\
c:\JCapper\_XML_2018
c:\JCapper\_XML_2019
c:\JCapper\_XML_2020
c:\JCapper\_XML_2021
c:\JCapper\_XML_2022

A few minutes ago (on 12-31-2021 at about 11:00 am Pacific time) I created the new folder for 2022 as shown above.

And tomorrow morning when I bring up Scratch BOT for the very first time in 2022, I'll perform the following steps:


2. With the new folder created, bring up Scratch BOT just prior to running your first XML Parse routine for 2022, select MENU, and select XML Folder Archive Tool.

The XML Folder Archive Tool should look familiar. It has the same UI (User Interface) as the DFM.

3. Use the UI to select and persist the new folder (created in step 1 above) where you want Scratch BOT to automatically archive your 2022 ScratchesAndChanges .xml files.

For me that will be the c:\JCapper\_XML_2022 folder.

That's It.

Happy New Year Everyone!



-jp

.


Reply
Reply
Copyright © 2018 JCapper Software              back to the JCapper Message Board              www.JCapper.com