Dataware House Architecture - 1947 Words

DATA WAREHOUSE ARCHITECTURE Albina Yusupova Data Warehouse Architecture The main difference between the database architecture in a standard, on-line transaction processing oriented system (usually ERP or CRM system) and a Data Warehouse is that the system’s relational model is usually de-normalized into dimension and fact tables which are typical to a data warehouse database design. The differences in the database architectures are caused by different purposes of their existence. In a typical OLTP system the database performance is crucial, as end-user interface responsiveness is one of the most important factors determining usefulness of the application. That kind of a database needs to†¦show more content†¦In view of this, it is far more reasonable to present the different layers of a data warehouse architecture rather than discussing the specifics of any one system. In general, all data warehouse systems have the following layers: ï  ¬Data Source Layer ï  ¬Data Extraction Layer ï  ¬Staging Area ï  ¬ETL Layer ï  ¬Data Storage Layer ï  ¬Data Logic Layer ï  ¬Data Presentation Layer ï  ¬Metadata Layer ï  ¬System Operations Layer ï  ¬ The picture below shows the relationships among the different components of the data warehouse architecture: Metadata Layer Data Source Layer Data Extraction Layer Staging Area ETL Layer Data Storage Layer Data Logic Layer Data Presentation Layer System Operations Layer Each component is discussed individually below: Data Source Layer This represents the different data sources that feed data into the data warehouse. The data source can be of any format - plain text file, relational database, other types of database, Excel file, etc., can all act as a data source. Many different types of data can be a data source:  · Operations - such as sales data, HR data, product data, inventory data, marketing data, systems data.  · Web server logs with user browsing data.  · Internal market research data.  · Third-party data, such as census data, demographics data, or survey data. All these data sources together form the Data Source Layer. Data Extraction Layer Data gets pulled from theShow MoreRelatedMobile Phones And Its Impact On Our Day Lives978 Words   |  4 Pagesjobs in Hadoop ecosystem. It is very difficult to develop the code and reuse it for different business cases. On the other hand, People are very much comfortable to query data using SQL like queries. A team of developers at Facebook developed a dataware house tool namely called as HIVE. Hive supports the queries like SQL type which is called as HiveQL. These queries are compiled as map reduce jobs and are executed using Hadoop. Through HiveQL we can plugin custom map reduce scripts into the queriesRead MoreUnderstanding Of Data Center Bridging1701 Words   |  7 Pagesavailability will definitely costs more, but it will be effective when we perform mistakes. Regular backups complement high availability by preventing data loss during high availability failure. High availability is usually maintained by creating architecture with data sources (DB), resources in two different data centers separated by a reasonable distance. In aws this is is called as avalibility zones (AZ). High availability DB’s are maintained by either mirroring technology or replication mechanismRead MoreBig Data Vs. Public Sector Organizations Essay2516 Words   |  11 Pagesperiod from 1990 till 2014 saw a tremendous growth in hard drives phenomena ranging from couple of hundreds of megabytes to 8 tera bytes and also the cost of hard drives relatively decreased. This is the main reason why we can say that concept of dataware house came in that is organization started considering the need of OLAP as compared to just the earlier relational databases, for analyzing the business and its customer needs and hence we can see huge amount of data getting generated these days as