Master files and transaction files
Transaction files and master files
Closely related to the idea of batch processing is the idea of a 'transaction file' and a 'master file'. Consider the following example. A company has many hourly-paid workers who are paid weekly. Each worker's personal details are stored in a computer record, which also holds details of how much that worker has been paid so far this tax year, how much has been paid towards their pension and insurance, how much tax has been paid and so on. All of the records about each employee are held in a nice organised order, using their Employee Identification Number. The first employee will be number 1. The second employee in the file will be number 2, and so on until the end of the file. This file (a file is a collection of records) is not accessed frequently. It is called the 'master file'.
- Each worker clocks on and off each day using a computerised system. Each time a worker clocks on or off, a record for that worker is accessed and updated. All the records that are frequently accessed and changed are known as the transaction file.
- At the end of the week, everyone's pay must be calculated and a payslip issued. To do this, a Job Control Language program is written. This lays out what files to use, when to use them, what program to run them with and what printer to send the output to.
- Both the transaction file and the master file are then accessed at the time laid down in the JCL. The details held in the transaction file are used to:
-
-
- Update the details in each employee's record in the master file.
- Calculate how much each employee should be paid and how much tax they should pay.
- Produce a payslip for each employee using the details held in the master file record for that employee and the record for that employee in the transaction file.
-
-
- The details held in the master file are required so that, for example, the name can be retrieved and printed on each payslip. Each record in the master file will also need to be updated with the new details about, for example, how much a person has been paid and how much tax they have paid to date.
Given that employees have been clocking in and out in a random order all week, the entries in the transaction file will all be in a different order to the nice order that the master file will be in. The first job that has to be done is to sort the transaction file entries so that they are in the same order as the master file, with all of the entries in the transaction file organised in ascending order by Employee Identification Number. If this didn't happen, it would take much longer to process a payslip because you would have to do a lot of searching for all of the entries for any particular employee throughout the transaction file once the master file entry for that employee had been retrieved. The entries for any particular employee will be all over the transaction file, which is why it takes more time than if they were all together and in the right order!
Once the transaction file is in the same order as the master file, the payslips can then be batch processed.
-
- The company's computers will be used in the middle of the night, when they are not needed for more important, more immediate tasks.
- The output from the batch processing isn't immediate - the payslips will be printed off and ready some hours after the processing has begun.
- The other important thing to note is that once the payslip processing has started, there won't be any need for human intervention. The processing will continue without help from a worker.
Backing up the master file and the transaction file
Once a particular week's transaction file has been used to print off payslips and update the master file, the transaction file is wiped clean, ready to be used for the next week's details. Interestingly, the only time when the master file is completely up-to-date is immediately after the master file has been updated. At any other time, the master file is out-of-date because it doesn't contain the details held in the transaction file. This means that if a company wants to ensure its data is backed-up, then it needs to ensure that both the master file and the transaction file are backed-up.
A shop example
Here is another example of batch processing. A shop sells many things. Each time a customer buys something, the purchase is recorded in time order in a transaction file. At the end of the day, the shop closes. The transaction file has many purchases throughout the day for e.g. Pepsi so the transaction file is first reorganised using a software program so that all of the Pepsi purchases are together (and indeed, all transactions for a particular product are together). This will speed up processing the transaction file because you won't have to constantly jump around the master file to different records when updating it. The master file, which holds a database of the products in the store at the beginning of the day is now clearly out-of-date because people have been buying things throughout the day. Using the transaction file and a software program, however, the master file can be updated. If there were twenty cans of Pepsi on the shelf at the beginning of the day, and there were five purchases for Pepsi during the day, then the master file can be updated to fifteen cans. When the master file is updated completely, it can then be backed up and the transaction file cleared, ready for the next day. Periodically, a manual stock taking in the shop has to happen, because the shop's master file will not reflect accurately what is on the shelf. This might be because of the problems from shoplifting or because products get damaged and removed from shelves without the database being informed.