What is Data Processing?

Data processing is the manipulation of data into a more useful form.

Data processing includes not only numerical calculations but also operations such as the classification of data and the transmission of data from one place to another. In general, we assume that these operations are performed by some type of machine or computer, although some of them could also be carried out manually. Data processing system refers to the equipment or devices and procedures by which the result is achieved.

There are various techniques of data processing, including manual, automatic and electronic data processing. 
  • Manual data processing is a traditional form of data processing that involves manual data collection and collection, analysis and presentation, such as bookkeeping and census exercises.
  • Automatic data processing is an almost obsolete data processing technique that was mainly done using electromagnetic devices, such as electric accounting machines and unit record equipment.
  • Electronic data processing is the widespread, modern technique of collecting, manipulating, analyzing and presenting data and information. Electronic data processing is also known as computerized data processing.
What are data processing methods?
Data processing methods are a way to turn large amounts of raw data into usable and understandable information, using methods such as batch processing, real-time processing, data mining and statistical processing. Data processing used to be done by hand and took a very long time, but today almost all data processing is done by computers, making the process much quicker.

Batch processing is the easiest form of processing, where computers analyze data in large batches at set times. For instance, payroll is batch data, because a computer analyzes all hours at the end of every pay period.

Real-time processing is used for data that can be analyzed immediately. For example, a radar system processes information in real-time, so that the operator gets feedback immediately.

Data mining takes information from multiple sources and tries to combine it and find patterns. For instance, grocery stores look at sales of one item and see if it correlates with another item. If everyone who buys peanut butter also buys jelly, they can put the two items together.

Data Processing Cycle
Once data is collected, it is processed to convert it into useful information. The data is processed again and again until the accurate result is achieved. This is called data processing cycle.

The data processing is very important activity and involves very careful planning. Usually, data processing activity involves three basic activities.

  • Input - It is the process through which collected data is transformed into a form that computer can understand. It is very important step because correct output result totally depends on the input data. In input step, following activities can be performed. Coding -- The verified data is coded or converted into machine readable form so that it can be processed through computer. Verification -- The collected data is verified to determine whether it is correct as required. Storing -- The data is stored on the secondary storage into a file.
  • Processing - The term processing denotes the actual data manipulation techniques such as classifying, sorting, calculating, summarizing, comparing, etc.
  • Output - Hope this doesn't require any explanation. 

Comments

Post a Comment

Popular posts from this blog

GP - Kerberos errors and resolutions

Issue:1. Ticket granting has expired with the error message, ERROR: psql: GSSAPI continuation error: Unspecified GSS failure. The Minor code may also display the error message, GSSAPI continuation error: Ticket expired. Solution: Create a new ticket on the Master server using the following command: kinit -k -t /home/gpadmin/gpdb-kerberos.keytab db_user Note: -t is the keytab file and db_user is the user. Issue:2. The library required to run the kinit command does not exist in the LD_LIBRARY_PATH  set in the Greenplum binaries, producing the following error message: ERROR: kinit: relocation error: kinit: symbol krb5_get_init_creds_opt_set_fast_ccache_name, version krb5_3_MIT not defined in file libkrb5.so.3 with link time reference Solution: Unset the LD_LIBRARY_PATH. IMPORTANT NOTE: The above fix will cause the following error when you try to run Postgres commands:  ERROR: psql: error while loading shared libraries: libpq.so.5: cannot open shared object ...
Read more

How to set Optimizer at database level in greenplum

How can I turn ON optimizer at Database level? You can set the optimier ON at database level using below command; ALTER DATABASE databasename SET OPTIMIZER = ON; Once this is done, kindly do reconnect to Database to see the change. How do I check if optimizer is set (ON/OFF) in a given database? You can use the show command as below to check psql -d databasename -c 'show optimizer'
Read more

GP - SQL Joins

Image
Available Join Types? Row Elimination Minimize Data Movement Join Implementation Seen in Query Plans Join Types: Inner Join Resulting data set is obtained from joining two tables on a common column Each row in left table is compared with each row in right table Matching rows will be present in result set Most commonly used Also know has Simple Join or EQUI-Join Left Outer Join Returns all rows from the left table even if there is no matching rows in right table. Will also get the matching rows Rows that are not matched in the right table will not be included in the result set Right Outer Join This is similar to left outer join, just need to flip the join Full Outer Join Will return all rows from both the tables where there is a match We will receive a NULL for rows which don’t match Cross Join Which is also called has Cartesian products Result set will have all rows on the left table paired with all rows on the right table Exam...
Read more