Serious Information

Who’s most gullible online and why? Secrets from scam world revealed

If you are like me, you have an opinion about this topic.  In fact we are often quite sure of what being “gullible” is like and how we would never be caught in the tricks and traps of the scam underworld.

But do we really know?  Are we really out of the “gullible zone”?

Instead of guessing, why not check out some of the findings in this report from the  security firm PC Tools and survey firm The Ponemon Institute .  I’m sure you’d find some of the results surprising.

CLICK on the following link: Who’s most gullible online and why? Secrets from scam world revealed.

© Rachel Agheyisi, Report Content Writer, Business Intelligence Notes Blog, 2012

Right Tools, Great Inspiration

Curious About Cloud Computing? Watch This!

It’s here, but yet unknown to many.  Cloud computing, that is.  So when I saw this video recently, I thought I should share it here as a way of spreading the word on the business benefits of  the cloud.  Stop by for more on the topic in the weeks ahead.

For now, CLICK HERE  and enjoy this informative, on-target take on cloud computing and document management from the friendly folks at NetDocuments.

How To Define and Select Good KPIs for Your Business

Introduction 

KPI (key performance indicator) is one of those buzzwords heard often in today’s business environment.  In theory, the meaning of the term is obvious enough.  However, for many companies, the implementation often poses a dilemma resulting in a pile of data with little or no actionable value.  That outcome may be due to a variety of factors, including a lack of clarity of the intent of KPIs.  This article offers a simplified definition of KPIs and suggests some criteria for selecting actionable indicators for your enterprise. 

Definition of KPIs 

A simple approach to selecting appropriate KPIs in any organization is to examine the three component words in the term: key, performance, and indicator. 

• “Key” suggests a leading principle; something of critical or central importance to the organization.  This means that not everything can be tagged as key – regardless of the availability and abundance of data.
• “Performance” suggests a process; an action recognizable to everyone (or at least the essential personnel) in the organization.  This means that the activity is public (within the organization or broader business arena).
• “Indicator” suggests a pointer; a gauge for a specific outcome in the organization.  This means that an indicator tracks change over time. 

Put together, KPIs are pointers that track outcomes of central importance to an organization. 

Typically, “outcomes of central importance” are formulated as business goals or objectives.  They embody the survival (success) strategy of the enterprise.  Hence, well-defined KPIs provide a basis for assessing the achievement of enterprise objectives.  Depending on the quality of the underlying data, KPIs allow for objectivity in telling a company’s success story. 

Few businesses are exempt from fluctuations in operations resulting from internal, industry, or economy-wide volatility.  Consequently, it is important to think of KPIs in terms of trends (over reasonably long periods) rather than focusing on them as a one-time measure.  It is also important to use the same definition of KPIs for the same objective to ensure consistency during the periodic performance reviews. 

Criteria for Selecting KPIs 

It is reasonable to expect that businesses in the same industry sector will have generally similar survival objectives.  However, the likelihood of operational differences means that KPIs are customizable for specific enterprise goals.  The point being that generic KPIs may not always be the best for your business. 

It is, therefore, important to have specific criteria for selecting KPIs that are consistent with the objectives of the enterprise. 

The ability to assess the components of performance constitutes an advantage in the use of key performance indicators (KPIs).  We suggest the following as desirable characteristics in good KPIs: 

1.  Strategic:  Put simply, the best KPIs originate from the central objectives of the organization.  This means that a good place to start the selection of KPIs is a thorough internal assessment of corporate objectives – particularly those considered critical to the survival of the enterprise.  The idea is to develop a set of questions or guidelines to help the company translate its objectives into actionable indicators.

An internal assessment is vital as a reference point for KPIs so that as enterprise goals change, KPIs can change accordingly.  Using a question format to develop the KPIs is effective in minimizing any fuzziness about the indicators and reinforces the links between KPIs and company objectives.

2.  Relevant:  Relevance provides meaning within the context of the organization’s hierarchy.  One of the most important reasons for tracking performance is to foster learning and improvement.  The logical approach to learn and improve is to design indicators that management and staff can identify with.

Relevance is vital to ensuring the informed participation of employees and management whose performance is critical to enterprise survival.  Additionally, relevance fosters realism in goal setting.  If the enterprise consists of multiple departments, which are engaged in multiple projects, relevance makes the monitoring of multiple goals manageable. 

3.  Quantifiable:  An enterprise mission statement is the place for lofty expressions.  KPIs, on the other hand, must be reliably quantifiable.  Measurement is what makes tracking meaningful for learning and improvement purposes.  Measurement is what adds intelligence and credibility to business decision-making.

KPIs summarize business strategy in ways that take the guesswork out of performance management.  The periodic (quarterly or annual) reviews allow for trend comparisons, assessment of set target levels, and systematic revision of goals.  Measurement minimizes subjectivity in the evaluation of individual employee and corporate achievement.  The ability to track KPIs encourages transparency in business performance metrics.

The Role of Good Data and Technology 

While the criteria for selecting good KPIs apply across all enterprises, the actual implementation is clearly a function of size.  The more complex the organizational structure and activities, the more complicated the management of business performance.  In all cases, good enterprise data is of paramount importance. 

Understandably, small companies can handle the choice of a few good KPIs, set up the data collection process, perform periodic measurements and reviews with relative ease. 

The more complicated the management structure, the more deliberation should go into the selection, measurement, and periodic reviews of KPIs.  The data requirement of medium- and large-size enterprises tends to be more than that of small enterprises.  This would explain why medium- and large-size companies are more likely to deploy IT support to facilitate the measurement and reporting of their KPIs. 

One such technology is business intelligence (BI) technology.  BI software (fully licensed onsite or acquired on a SaaS platform) provides tools for gathering, coordinating, and transforming data so that information is derived efficiently.  The more integrated the BI system, the more functionalities it offers, including data management tools, reporting tools, and dashboards for viewing and sharing results. 

An Example of KPIs for a Professional Services Consultancy 

The survival of a professional service provider depends on the number of ongoing and new projects the company handles.  Specifically, the company’s gross revenue is primarily determined by the billable hours it invoices.  KPIs for such an enterprise must be strategically linked to target growth in number of projects (not necessarily number of clients) and time billed by consultants. 

The indicators could be refined to include target levels of growth in billable hours for each category of consultants, profit center, department, and geographical location.  Data needed for tracking this type of KPI are contained in employee timesheets, which show hours billed by project.  It is relatively easy to compile the information necessary for assessing progress toward set targets, which in turn provides the gauge for company revenue growth. 

Incidentally, the professional service provider could be a large multi-national entity or a single proprietorship.  The process of tracking revenues based on billable time is similar – the main difference is the amount of data involved. 

Conclusion 

Realistically, KPIs are seldom perfect measures.  However, their usefulness for tracking organizational performance hinges on how well the indicators connect with the central objectives of the enterprise. 

Ensuring that the selected indicators are strategic, relevant, and quantifiable improves the likelihood that they will foster realism, objectivity, and transparency in performance management. 

IT support may facilitate the measurement and reporting of KPIs, especially in enterprises with complex organizational structure and activities.  Done manually or with the help of technology, good data are crucial to meaningful KPIs. 

 

© Copyright Rachel Agheyisi and Business Intelligence Notes Blog, 2009.

Is SaaS right for your business?

SaaS stands for software as a service.  SaaS-based tech products are marketed as “hosted solutions” or “web based solutions”.  Providers of SaaS-based BI solutions, such as SAP BusinessObjects and SAS, refer to the platform as OnDemand.  The platform typically combines two features:  a service provider and the Internet.  The purchaser is typically referred to as end-user (rather than owner). 

The obvious advantage of SaaS is that regardless of size, any company can potentially afford the best and latest business technology.  The upfront costs of SaaS access are considerably low in comparison to costs associated with licensed software and purchased equipment. 

SaaS is receiving increased attention, especially in the current economic times characterized by budgetary constraints.  However, the service model underlying SaaS is not altogether unique. 

Somewhat similar arrangements can be found in other industries, especially those that sell high-ticket items.  For example, the automobile industry offers leases that allow qualified customers (often based on credit rating and driving record) to enjoy the benefits of transportation at a fraction of the cost of outright purchase.  In the aviation industry, fractional ownership makes air transportation via corporate jets more affordable.  Time-sharing arrangements in real estate allow multiple co-owners to enjoy the comforts of home (such as vacation homes) at a fraction of the cost. 

After much internal debate, fueled by comprehensive research, I finally opted to lease a brand new car a few year ago.  The process was so painless; I started wondering why I waited so long.  Not anymore.  Now, I can drive a new car every other year, by simply “trading” one lease for another.  Better yet, I can trade up car models, if I want.  Of course, it helps that I live in California where the auto leasing process has been so streamlined, you don’t need a law degree to figure out the terms.  Additionally, my driving preferences (annual mileage, type of car, dealership service, etc) are best served by the kind of fractional ownership offered by a lease. 

What all of these arrangements have in common is affordable access, also known as lower total-cost-to-own (TCO).  However, they also come with terms, conditions, and restrictions that may not be suitable for everyone.  Therein lies the rub. 

This brings me to back to SaaS.  Is it right for your business?  The simple answer is it depends —-on various factors.  This article looks at some key issues to help you assess the suitability of SaaS for your business software needs. 

Your business needs first 

This may sound like a no-brainer.  However, it is worth stating.  The best place to start with the decision for or against SaaS is with your current and prospective tech needs.  The temptation to look first at products and service providers is strong.  But starting with your business needs creates the best context for the choice of product and service provider.  Here are a few suggestions for handling this phase: 

• Internal brainstorming:  Put your business in a position of strength by doing the necessary internal brainstorming, and a thorough needs assessment, including an inventory of current tech, usage, deficiencies, and redundancies.
• Short-term or long-term tech need:  What are you in the market for?  Determine if the tech product in question serves essentially short-term business need or if it is customizable as your business evolves.
• Compatibility:  Is the technology compatible with what your core clients are using?  Will deploying the latest and the best software create integration/transition problems?
• A point person:  It is helpful to appoint a point person to handle your business interaction with the (prospective) service provider.  If possible, request that the provider designate a representative who is knowledgeable about your industry.  A good liaison fit ensures a smooth adoption process and is vital for when you need answers to your “tough” questions.
• Prepare to negotiate:  It is OK to negotiate — everything, including set up fees, “hidden” fees, renewal terms, upgrades, and service alerts.  Prior preparation makes this possible.  Prior preparation should include, at a minimum, some comparison data for key service providers. 

Understandably, some of these issues will be less significant as SaaS platform permeates more tech applications.  However, they will continue to be important to a successful SaaS implementation, one that aligns with your business goals. 

Cost Considerations 

Cost is probably one of the first things that decision makers consider prior to investing in a new technology, especially one that is relatively expensive.  This is both practical and understandable concern.  SaaS allows the end user to deploy business technology at relatively lower price without the costs of hardware, updates, and maintenance.  It is easy to see the appeal of SaaS compared to costs associated with licensed application. 

However, upfront cost should not be allocated a disproportionate weight in the decision matrix.  This is particularly important for small businesses to take a comprehensive view of cost when considering SaaS.  Inexpensive does not always translate to quality, neither does expensive mean best fit. 

Depending on the terms of the contract, the long-term cost of SaaS access may include more than the subscription-related expenses.  Additionally, the tax implications of SaaS versus the conventional platform need to be factored into the cost consideration.  Typically, there are tax advantages associated with licensed software and business equipment.  Those advantages are generally eroded under SaaS. 

Data Security and Regulatory Compliance Standards 

The convenience of a web-based, hosted solution is a desirable benefit of the SaaS platform.  It allows users easy access to business databases from anywhere they have Internet connection.  Data sharing, including from multiple geographical sites is expedited. 

However, with web-based access come concerns about hackers, data theft, and compromised privacy rules. 

Securing your business data and ensuring compliance to relevant regulations are vital business functions.  In fact, U.S. regulatory agencies, such as the Federal Trade Commission (FTC), hold each business accountable for the security of personal, sensitive information in their possession – even if the information is accessed and processed by third parties. 

This means that beyond the asset value of business data, there are real legal reasons to be concerned about the integrity of SaaS-related connectivity.  It adds weight to the phrase know your business partners.  In this context, it means it is your responsibility to ensure that the SaaS solution provider complies with current appropriate security policies and procedures. 

Contract Terms 

I’m not a lawyer, but once you sign it, you agree to the terms and conditions of a contract.  In short, read and endorse with understanding.  This is vital because a badly written contract (from your business point of view), can translate into business interruptions, unexpected costs, and other undesirable consequences. 

As with all contracts, a thorough understanding of the pricing terms, escalation clauses, renewal, upgrades, penalties, connectivity issues, unplanned downtimes, and other required and optional service arrangements is crucial for beneficial SaaS implementation. 

The easiest way to preempt contract-related surprises is to read the contract document in its entirety.  This may be time-consuming (in cases where the document is very long), but it will be time well spent.  On the other hand, if the contract is long and filled with legalese, it might be wise to seek the input of an attorney or look for another provider with less complicated contract language. 

Conclusion 

Few businesses can survive in today’s economy without current tech-based business tools.  Relevance and affordability are important considerations in tech deployment.  In tough economic times, budgetary concerns take on added importance.  The SaaS platform promises reduced total cost to own, which creates a high appeal for SaaS tech solutions. 

To determine if that appeal applies to your business, it is important to conduct a thorough internal needs assessment, and take a comprehensive approach to pricing/cost.  Appropriate security, and a clear understanding of the bold and fine prints of the service contract are also crucial to a successful SaaS implementation.

 

© Copyright Rachel Agheyisi and Business Intelligence Notes Blog, 2009.

How Does Data Mining Work?

In an earlier article, we introduced some basic features of data mining.  At its core, data mining tools reveal data relationships that can transform business processes.  Although the specifics may differ, practically all data mining software operate on the same premise:  develop data groupings based on identifiable attributes. 

This article examines some of the groupings used to support  business decisions.  It also highlights components of data mining models. 

Data Relationships 

A clear understanding of the data is important for determining how to exploit the information for business purposes.  One way to understand data is to create one or more of the following related groups: 

1.       Data Classes 

Data classes are groups that share easily identifiable characteristics.  This explains why they are also referred to as predetermined groups.  In the context of a retail business, customers who have purchased a particular product constitutes a data class. 

For example, Amazon.com customers who have purchased business books in the past constitute a class.  Knowing the characteristics of the data class takes the guesswork out of “likelihood to buy” factor in sales promotion.  The online retailer can use this grouping to develop marketing campaigns for business books and target customers in the group (and underlying sub-groups).  Depending on the size of each class, data grouping can significantly improve the efficiency of mass marketing. 

2.      Data Clusters 

Data clusters are similar to classes, but include additional attributes such as logical relationships.  In the context of business applications, consumer preferences are often the most useful attributes.  Consumer preferences can be used to understand market segments and customer loyalty.  Accurate clustering can support cross selling. 

Again, using Amazon.com as an example, data clusters allow the retailer to identify what other products are purchased by customers who buy business books.  Armed with this information, the retailer can develop “product recommendations” as part of its customer relations management (CRM) programs.  The ability to nurture leads efficiently is critical to sales. 

3.      Data Associations 

Data associations take clusters further.  In the context of business application, associative data mining reveals buying patterns that would otherwise go unnoticed.  For example, changes in buying habits induced by shifts in the economy require in-depth analysis for accurate characterization.  A clear understanding of the economic shifts can be exploited for marketing purposes. 

For example, an economic recession triggers a variety of changes including widespread corporate downsizing, increased unemployment, spikes in self-employment and home-based businesses.  All these changes translate into shifts in business purchases.  Data mining technology can help track associative relationships among these shifts, which in turn can inform marketing strategies. 

4.      Sequential patterns 

While analyzing past purchases is helpful, some experts believe that the true benefit of data mining is to anticipate customer purchases through predictive analytics.  By building on historical data, sequential patterns allow projections to be developed.  The projected industry trends are essential for forward-looking business planning and competitive intelligence.

Components of Data Mining Models 

Without exception, the potential benefits of data relationships hinge on the quality of the underlying data.  The well-known saying that “garbage-in-garbage-out” applies.  Inaccurate or improperly coded data will produce inaccurate and misleading groupings, which in turn will produce wrong business signals. 

In addition to data quality, the functionality of a data mining tool depends on the underlying modeling methodology.  Practically all data mining models advocate careful data collection, data preparation, evaluation, and analytical controls. 

For example, the data mining solutions developed by the SAS Institute incorporate a sequence of steps dubbed SEMMA, which stands for Sample, Explore, Modify, Model, and Assess.  According to SAS, its modeling methodology streamlines the data mining process, enabling business analysts and data miners to create highly accurate predictive and descriptive models based on analysis of vast amounts of data from across the enterprise. 

Predictive accuracy is vital.

Conclusion 

Few will contest the potential of data mining tools to create valuable business insights.  However, as with all technologies, the deployment of data mining needs to be driven by well-researched enterprise needs, as well as cost and usability considerations.

 

© Copyright Rachel Agheyisi and Business Intelligence Notes Blog, 2009.

Mining Data for Business Insights

Anyone who has an Amazon.com account has experienced the power of data mining at the retail level.  The company periodically sends its regular customers “recommendations” of products they might like based on what they (or like-minded consumers) bought in the past.  It is classic cross selling made easy.

 How does Amazon.com know what you might like?  The simple answer is the company has deployed data mining software that reveals vital information about your buying preferences.  This article looks at some features of data mining technology.

What is Data Mining? 

Data mining is a powerful tool for digging deep into enterprise data to reveal underlying patterns and relationships that can be used to build prediction models.  The tool brings the benefits of predictive analytics to business processes.  In short, data mining fuels business insights through trends predicated on detailed analysis of vast amounts of related data. 

This explains why data mining tools are worth considering in the context of a database or data warehouse and business intelligence (BI) system.  It also explains the trend among  power users toward the integration of analytical functions into a data warehouse.    

What Types of Data Can Be Mined? 

Generally, when people talk about data mining, they talk in terms of numbers.  This is understandable as practically all initial deployment of data mining technology was directed at sifting through numerical data, including: 

• in-house transactional data, such as sales, purchases, and inventory;
• in-house operational data, such as payroll and administrative data;
• third-party data, such as industry trend data, pertinent national and global forecast data. 

However, businesses now accumulate vast amounts of non-numerical information, in the form of text-based documents, emails, survey feedback, and meta data (a.k.a. data about the data itself).  The explosion in the business role of the Internet translates into significant amounts of non-numerical data captured from diverse Web-based applications.  These text data are a mixed bag, which must be sorted for a better appreciation of their information value.  Therein lies the relevance of text mining tools used to analyze text documents by extracting key phrases and attributes, which in turn can be combined with other data to create a more comprehensive picture for decision-making.  

 Who Are the Big Users of Data Mining? 

Who knows your credit card better than you do?  Your favorite retailer, that’s who!  In addition to Amazon.com, other major users of data mining tools include Wal-Mart, major credit card issuers, and retail supermarkets.  

These companies design products and promotions to target customers identified by buying behavior, demographic characteristics, and income.  Even your geographic location becomes crucial marketing information when combined with other seemingly disparate bits of data about your buying habits. 

Because of the capability of data mining technology to drill through layers of data, it provides insights at almost granular level.  Information can be distilled to permit the configuration of stores and in-store product displays to capture specific customer segments.  It takes most of the guesswork out of designing effective marketing campaigns. 

In much the same way that data mining can be used to identify patterns that drive efficient marketing strategies, it can also be used to flag data anomalies, which can be helpful in fraud detection and prevention. 

Leading Data Mining Software on the Market 

Recent industry surveys from IDC, Datamonitor, and Forrester Research put SAS Institute and Oracle at the top of the list of providers of standalone and integrated data mining tools.  Other vendors making the top providers’ list include familiar BI brands, such as  IBM-Cognos (soon to include SPSS), SAP-Business Objects, Microsoft, and Information Builders.

 © Copyright Rachel Agheyisi and Business Intelligence Notes Blog, 2009.

Data Warehouse in the BI Neighborhood

A solid understanding the workings of a business intelligence system requires looking at the constituent parts.  In this article, we examine the data warehouse component.  Consistent with the underlying complexities, there is a variety of data warehouse definitions in the literature.  However, the complex issues are saved for subsequent articles.  First, we cover some introductory and foundational material. 

What is a data warehouse? 

As the name suggests, a data warehouse is a data storage system.  The data can be massive in volume, originate from multiple systems, and be very detailed (granular).  This allows stored data to be manipulated, analyzed, and presented in diverse ways.  Terms, such as scrub, slice, dice, group, aggregate, mining, are used commonly to describe operations undertaken in a data warehouse environment.  However, what a data warehouse is in reality is not always linear due to the nuances introduced by individual software developers. 

If designed primarily for use in a BI environment, a data warehouse typically contains historical data derived from multiple operational, transactional and non-transactional systems.  The ultimate goal is to identify meaningful relationships among the various data to inform better business decisions. 

What is data warehouse architecture? 

The architecture of a data warehouse describes its structure, including how the components fit together.  Not surprisingly, there is no single, universally applicable architecture.  The academic debate of what constitutes the best architecture dates back over a decade.  Anyone interested in the historical roots of the issues should read the seminal works of Ralph Kimball and William Inmon. 

Stripped of individual developer’s labels, there are three discernable architectural structures: 

1. Basic architecture.  This is, structurally, the simplest warehouse design.  It allows end users to have direct access to data from various source systems through the warehouse.
2. Basic architecture with a staging area.  This design is similar to the basic structure, but includes a staging area.  The staging area serves many functions, including space for data cleaning, reformatting, and organizing the incoming data before loading into the warehouse.
3. Basic architecture with staging area, and data marts.  The inclusion of data marts represents the type of customization to improve the fit of a data warehouse to each organization’s needs.  Data marts contain operational data that serve the specific needs of designated business users in terms of content, presentation, analysis, and schedule. 

CLICK HERE for a graphical depiction of an integrated data warehouse architecture. 

What factors should be considered before investing in a data warehouse? 

Software developers expound the desirable attributes of their offerings, which typically target specific industry segments.  Ultimately, the choice of data warehouse architecture is (or ideally should be) dependent on the specific needs of the enterprise making the investment.  In addition to fit, planning, implementation duration, ease of use and maintenance, and costs are all important considerations in the choice of data warehouse architecture. 

Here are some tips to include in your planning checklist: 

• Involve all key stakeholders in the discussion of the organization’s need for a data warehouse before you proceed.
• Brainstorm on factors that could influence that decision, such as specific business requirements, scope of the application (across the enterprise or localized) and type of architectural design.
• Develop a strategy on how your preferred design might evolve over time.  For instance, if cost is a concern, explore options on how best to phase in an integrated warehouse design.
• Shop around for the vendor whose data warehouse platform and service terms match your needs.  The list of top brands is short and includes the who’s who of the tech world like Oracle, Microsoft, and IBM. 

What are typical data warehouse operations? 

Depending on design, a data warehouse supports end-uses such as: 

• querying, the results of which are typically structured as tables
• reporting, which can be delivered in customized formats suitable for general and specialized end-users , and
• analysis, which allows for a range of functions that capitalizes on the data mining capability of an integrated data warehouse. 

Conclusion 

As with all decisions about technology deployment, it pays to do a thorough needs assessment.  That cautionary point is especially important in decisions relating to a data warehouse.  The more integrated the warehouse architecture, the more resource-intensive it tends to be, and the greater the need for objective pre-deployment planning.

© Copyright Rachel Agheyisi and Business Intelligence Notes Blog, 2009.

ETL: in the business intelligence neighborhood

Business intelligence (BI) is a tech-based tool that allows decision makers process information to gain insights that drive enterprise performance.  However, the path from information to knowledge is not always linear.  Typically, a series of steps, including collection, organization, evaluation, and transformation precede the data analysis and reporting that provide insight. 

Issues relating to data access – the so-called ETL components – have significant implications for practically every phase of a BI project.  ETL stands for extraction, transformation, and loading of data. 

The implementation of ETL depends on factors such the type of data, number of data sources, data volume, and the preferred mode of data transportation.  This article highlights ETL in the context of a BI project. 

Extraction 

Extraction involves identifying the relevant data sets and pulling them from one or more operational and transactional source systems.  The size or volume of the data is a function of business operation.  It is not unusual for businesses to generate huge megabytes of data from multiple sources on a daily basis.  The choice of extraction method depends on the specifications of the data source systems and the target warehouse environment. 

One of the challenges of the extraction phase is making sense of possible differences in the formats of the data source systems.  Examples of commonly used formats include flat files, relational databases, and non-relational databases.  The variety of formats necessitates the parsing of extracted data to determine their cohesiveness and conformity with underlying structures. 

Other considerations in the extraction phase include whether or not to incorporate data transformation as part of the process, whether to extract the data directly from the source systems, or extract them from a staged area outside of the original source systems.  None of these considerations is trivial. 

Transformation 

The transformation phase of ETL involves preparing the extracted data for loading to the target database or databases.  Some data may require little or no transformation, while other data sets will need to be restructured, cleaned, and organized before transportation.  Typical types of data transformation include: 

• Re-mapping
• Sorting
• Filtering
• Cleaning
• Aggregation
• Pivoting
• Recoding
• Standardization 

SQL functionality makes several of these transformations relatively routine in most BI environment. 

Loading 

Experts consider data loading from the source systems a simple task in the ETL process, assuming a successful completion of the extraction phase.  Depending on the extraction method, data loading may overwrite existing data, be cumulative, or involve a blend of both.  Each requires specific constraints, triggers, and audits to ensure the preservation of the integrity of the source data. 

The most direct loading route involves moving the extracted data from a source system directly to a target database in a data warehouse.  Other less straightforward transportation routes involve an intermediary (pre-warehouse) staging database or an additional (post-warehouse) database.  This explains the evolution of ETL tools, which are intended to facilitate the processes.  Practically, all the top BI brands include ETL tools. 

Conclusion 

The foregoing are highlights of the ETL processes in a BI project.  Although relatively easy to compartmentalize, the actual implementation of ETL varies from enterprise to enterprise.  In practically all cases, ETL is resource-intensive.  Any enterprise seriously considering BI deployment needs to commit significant amount of time to shop around. 

White papers will prove to be helpful research resource for the quick education of current and prospective BI users.  Most vendors post these reports on their websites.  However, less focus on the flashy side of BI will prove beneficial in identifying what is ultimately useful for the enterprise. 

 

© Copyright Rachel Agheyisi and Business Intelligence Notes Blog, 2009.