Prune days and Change Capture in DAC

Here is a question I often get from customers regarding change capture by DAC when it is used for extracting Siebel data. The problem is equally valid for non-Siebel data sources if the default change capture mechanism provided in DAC is used for identifying changes in source data.

How can we ensure that data committed with an update timestamp between the start of an ETL run and the end of the extract is not excluded by the next ETL?

• DAC stores a value called LAST_REFRESH_DATE at the end of an ETL extract.
• During the next run, DAC captures changes to the source data by comparing this LAST_REFRESH_DATE with the LAST_UPD date of records in the source.
• So if the LAST_REFRESH_DATE is less than LAST_UPD of a source record, then the record being processed has either been updated or created between the time of last execution and the current run
• But what will happen if a record in the source is committed between the time of starting the data extract and completing the extract with a commit date that is less than the LAST_REFRESH_DATE

Prune days to the rescue!!! The solution to the above problem is to set the PRUNE_DAYS parameter.

If the prune days parameter is set to 1 for example, the change capture process will use LAST_REFRESH_DATE – Prune days as the cutoff timestamp for change capture

Here is an example of such a scenario

• Let us assume that there is a table called S_CUSTOMER in transaction database with 2 records created during business hours on the 1st of January.
• On the January 1st, Extraction of Customer Data begins at 1700 hours and completes at January 1st, 1800 hours into a customer Dimension called WC_CUSTOMER.
• LAST_REFRESH_DATE stored is January 1, 1800 hours


• Now, a new customer is created between 1700 and 1800. But it is committed to the database AFTER the extract is completed with a timestamp of January 1st, 17.30



• More customers are added to the transaction database during the next working day. As a result the following is the condition of the source table at the time of the next ETL Extract i.e. Jan 2, 12:00 am





• Now the change capture process in DAC will compare the stored LAST_REFRESH_DATE with the update timestamp of the records in the source data and pull only the records updated after this LAST_REFRESH_DATE i.e. Jan 1, 1800 hours into the target table. Unfortunately for the 3rd record depicted below, the LAST_UPD is Jan 1, 1730 i.e. it is before the LAST_REFRESH_DATE. So, it does not qualify for the incremental extract and we have a missing record in the target table.



• g) But if we had set the PRUNE DAYS parameter to 1, the cutoff date used to identify changes in source data would be .Dec 31, 1800 hours i.e LAST_REFRESH_DATE – 1 day . If this ploy is used, all the records with LAST_UPD > Dec 31, 1800 hours will qualify for the incremental extract and we will have a complete set of data in the target table

The Prune days parameter can be used to ensure that DAC Change Capture does not miss out data created around the time of data extraction.

Set it with care to a reasonable value to avoid increasing the volume of data selected by incremental load process.

New Feature in DAC 7.9 for non-Siebel Sources

For source systems other than Siebel, the Prune Days setting is used in the same way except that the DAC subtracts the number of prune days from the LAST_REFRESH_DATE of a given source and supplies this as the value for the $$LAST_EXTRACT_DATE parameter.

Inputs by: Raghunatha Yadav & Sanjay Rao

Read More about DAC

A test drive with the Siebel 8 ST engine

A new script engine i.e. the Siebel ST eScript engine has been made the default scripting engine in Siebel 8.0.

In case you did not know, Siebel eScript is a scripting or programming language that application developers use to write simple scripts to extend Siebel applications. Till recently, the default script engine for Siebel Tools and Client Applications was the T eScript engine. And ST for the uninitiated stands for Strongly Typed.

If you stayed awake through your programming classes, you might remember that strongly typed languages bind variables to a particular data type. So if you declare a variable to be of type Dog, you cannot abruptly decide to assign a Road-Roller to the variable midway through the program. The good thing about this is you cannot call invalid methods like Road-Roller.bark() or Dog.BlockTrafficAndAnnoyCommuters() by mistake.

The bad thing is that you need to declare more variables and therefore need more runtime memory when using a strongly typed language instead of a weakly typed languages. There are proponents for both types of programming languages. But that is not the issue that interests us here.

What did interest me is that Siebel claims that the ST eScript engine provides the following enhancements:

• Improved performance and self documentation due to strong typing of variables

• ScriptAssist utility and

• Fix and Go feature

I decided to take the ST engine out for a test drive to see if the above claims are indeed true.

In my little experiment, I decided to use a COMCreateObject to pull the data from an external source, in Excel and update the Siebel database. The excel file contained some new records that and some changes to existing records. The update would be triggered on the click of a button which would in turn invoke a Business Service which comprises of many functions.

Here are my findings:

ScriptAssist

I loved the new Script Assist Feature which makes the life of the programmer much easier providing auto-complete, auto-indentation, method listing, and method signature capabilities.

There is no longer a need to remember all the methods related to an object. Just press Ctrl+Space or “.” and ScriptAssist will display a dropdown of all methods and properties available for that particular object.

This is a big timesaver and is highly appreciated by us.

Auto-complete is another nifty feature that helped us avoid a lot recompilation. Just enter the first 2-3 alphabets of a method and it shows the available method name. For example, when I started off with the letters “BusC”, the editor offered to save me the effort of typing in the remaining 5 letters in “BusComp()”. This is similar to IntelliSense technology from Microsoft.

Fix and Go

With Fix and Go enabled, there is no need to compile the .srf to restart the debugger after making a change.

This was a big pain in the bad old days. With fix and go, I hope to have a lot more time to spend on “strategic meetings” near the coffee machine.

Performance improvement because of Strong typing:

I tried to see if I would get any performance improvements by using ST Script by putting an equivalent piece of code created in ST-Engine with another piece of code in Siebel 7.8 T-Engine

Although we ran the code through some 10,000 iterations, we did not see a major difference in the execution time.

However, there should in theory be a performance improvement on account of saving time on type-checking at run-time and compile time.

Also, the code should be more robust as it would not be possible to call invalid methods as it would be in a weakly typed language

So, should I throw away all my T-script Engine code and embrace the new ST-Script Engine ?

It depends on the amount of scripting you have done in your Siebel Installation. If you have heavily customized your objects with scripting, it is going to be a major exercise to verify that the code is working correctly in the new environment.

But once, the upgrade is done, there should be a performance improvement and your application will be more robust because there is a lower likelihood of getting those errors because of calls to invalid methods.

Inputs from Sajid Attar and Abhinav Anand.

Read More about Siebel 8.0

Siebel Smartscript

Business Scenario
Call center/helpdesk agents have customer interactions, in which they address customer issues, ask questions and resolve problems. The chances of the customer coming back become higher if the interaction has been satisfying. Thus, in order to serve the customer better, capturing the flow of interaction becomes vital. In such real time scenarios, Siebel Smartscript helps call center agents to define application workflow and capture customer interactions.

Siebel Smartscript and its usability
Siebel SmartScript guides agents through each customer interaction, suggesting solutions based on the customer’s profile, environment, current requirements, and buying patterns. Siebel Smartscipt GUI includes questions in the right pane and tree structure summarizing the questions and answers in the left pane. In order to build this UI developer needs to include the script, page(s), question(s) and answer(s).

The answers of Smartscript can be stored in a BC, few values can be updated, and some business logic or functionality can be extended based on these answers. For such customizations script can be written at question or script object level. Smartscript programs can be coded in VBScript or escript. Some additional syntax has been provided by Siebel only specific to Smartscript usage.

Why Siebel Smartscript?
It is user-friendly having a very intuitive GUI. Programmers having basic knowledge of coding can graphically create Smartscript. The scripts written for pages, questions and script objects can be re-used and can have multiple language translations. Using SmartScripts the customer interactions become more personalized and effective in nature.

Alternatives to Siebel Smartscript
Using customized applets and embedding scripts is an alternative to Siebel Smartscript. This alternative has demerits of increased development time and cost, weaker GUI and upgrade difficulties.

New features in Siebel Smartscipt 8.0
In Siebel 8.0, Smart script can be invoked using a hyperlink. Also in the newer version Business services and Siebel Assignment manager can be invoked from Siebel Smartscript. Siebel Smartscript 8.0 has an improved GUI too.

Inputs from Shardul Trivedi

Read More about Siebel Smartscript

Single Sign-On in Siebel

Being a software developer in Siebel, I had numerous opportunities to create something using this tool. Apart from acknowledging client requests which is everyone’s number one priority, like any other developer I also wanted to first get my fundamentals strong in this tool. There were times when I tried 2-3 different ways to deal with one of the old demands and was successful in discovering many new and simpler methods. Since then I always experiment to meet the requirements and never hesitated to play with Siebel.

This led me into using these concepts readily and implementing them in real time. While using Siebel for many servers in my project, the traditional practice to Sign-In every time you open a new Siebel application server was not only exasperating at times but also a shortcoming in terms of Siebel’s user friendly behavior. Searching an answer for this problem I came across the Single Sign-On Process.

For starters, what is Single Sign-In (SSO)?

It is a session/user authentication process that allows a user to enter name and password for only a single time in entire session in order to access multiple applications. The process authenticates the user for all the applications they have been given rights to and eliminates further prompts when they switch applications during a particular session.

Single Sign-On (SSO) is becoming the standard for advanced security in IT sector. This equips us in streamlined, consistent and secure password management policy for every user. Cutting down on the number of passwords issued to users and reducing time spent on administration, support and training, SSO directly impacts the daily user ability.

The following steps need to be followed for configuring Web Single Sign On and the Active Directory Security Adapter.

1. An ADSI service account will be needed.This service account is one used to bind to the Active Directory and will also hold the shared database credentials.
2. The service account will also be the web anonymous user for Single Sign On and must have a Siebel login with Web Anonymous User responsibility. This anonymous user does not need a database account.
3. Create the shared database account. All users will connect to the database with this database account instead of an individual database account.
4. Create a custom application object manager that will be configured for ADSI security adapter authentication instead of database authentication.
5. Add the SSO Sales Object Manager to the Siebel Web Server Extension Update the Siebel Web Server Extension eapps.cfg file to include the SSO Sales Object Manager.
6. Add a new virtual directory to IIS for SSO.
7. Configure Siebel Web Server for Web SSO. Open the eapps.cfg file and set parameter IntegratedDomainAuth = True in [swe] section and add following parameter in [SSO] section.

- SingleSignOn=TRUE
- TrustToken=token name
- userspecsource=server
- userspec=REMOTE_USER
- StartCommand= SWECmd=GotoView&SWEView=Sales+Home+Page+View&SWERF=1

This will enable the Single Sign On and it will be authenticated by Windows Integrated Authentication.

Configure ADSI Security Adapter Named Subsystem in Server Administration – Configuration; Enterprises; profile configuration tab and set the parameters for ADSI.

If the authentication is stolen then all resources becomes vulnerable. In addition, you need to be aware of whether there are backups to the SSO in the event the system is down; if the Server is down then the network resources will remain unavailable.

Post by Suman Kumar

Read More about Siebel

Scripting Limitations in Siebel CRM

There are nearly 200 different types of objects that can be configured in Siebel to reflect the changes in the User Interface Layer, Business Objects Layer and the Data Objects Layer. Being a Siebel Professional, I often come across complex business Requirements that cannot be met solely by configuration. Siebel Scripting is great for handling these complex needs. But there are a few limitations to be considered while using scripting in Siebel development.

• 16 KB Size limitation
  In Siebel eScripting, a single method can contain only 16KB of code. But in real life, one may have to go in for heavy customizations of Siebel objects for which 16KB is just not enough. My workaround in such cases is to split large methods into smaller methods.
• Cannot mix scripting languages in an object
  Most scripting in Siebel is in eScript. But, sometimes we need to use third party system/objects like ActiveX/COM where VBscript is the preferred language. So how do we get to external objects like ActiveX/COM objects from a Siebel Object coded in eScript given that we are not allowed to mix scripting languages in an object?. The solution is that we call another Siebel Object which is scripted in VBscript through which we can finally access the external system/object. This solution works fine. But it does lead to performance issues. Better ideas are welcome
• Dynamic switching between server and browser side script not possible.
  We can call server script from browser script. But once server script starts executing, it is not possible to call browser script. So, it is not possible to do things like call pop-up a message on the browser side after executing say 10 lines of code on server side. If anyone has any workarounds, please do let me know.
• Object has to be compiled after every modification.
  Till Siebel 7.8, developers had to compile and re-compile modified objects a every minor change in code. This is a big pain while debugging and a major time-waster. But in the latest version 8.0, Siebel provides “Fix and Go” option with ST Engine which has addressed this limitation. Thank you Siebel!!!
• Scripts are not always upgradeable.
  Scripts often give errors while upgrading a Siebel Application from one version to another. At times we have had to comment out entire sections so that the technical upgrade can complete. Finding all these problem areas in the script and fixing them is a big challenge during upgrades.

Feel free to comment/discuss/debate!

Read More about Siebel CRM

Analyzing Analytics

Dashboard Power
The other day, I was ‘Googling’ for a Siebel Analytics document when I stumbled upon this article which is quite a good example of how the prowess of Analytics is making life easier for commuters. Siebel Analytics now rechristened Oracle BI, helped officials at Manchester airport to cope with the extra pressure of screening additional bags by identifying future trends.

It provided Manchester Airport’s management team with interactive dashboards of flight, passenger and baggage movements. As a result, Manchester Airport lifted the ‘One Hand Baggage Rule’. The full article can be found out at the following link: http://www.computerweekly.com
/Articles/2008/01/07/228771/manchester-airport-lifts-one-bag-rule-with-oracle.htm

It’s a bit of wishful thinking but imagine the ease in travelling if every airport harnesses the capabilities of Siebel Analytics and lifts the above mentioned rule.

This first post of mine in the ‘Analyzing Analytics’ series is an attempt at exposing the readers, be it a neo-techie or a seasoned expert, to the vast capabilities of the end product of Siebel Analytics: The Intelligence Dashboards and the concept of Dashboard centric business operation and management. So let’s discuss what these ‘Dashboards’ are all about.

Case 1: What are the sales for this quarter? What’s the comparison of Sales for this quarter with year ago sales? How many customer accounts are in the red? Such kinds of questions are very common in day to day business analysis. The answer to these and a plethora of similar process oriented questions helps in running a business efficiently.

Case 2:Organizations often have data fragmented across multiple sources like spreadsheets, department specific systems, etc. Often critical data is buried deep somewhere in some file or system in the organization.

Keeping the above two cases in mind, a unified view of the organization is the need of the hour. An Analytical dashboard is just what any organization would desire. It helps in giving a holistic view of business operations, most often using data that has been gathered into a warehouse and occasionally working from operational data. Pre requisites to a healthy dashboard are clean data, a good data model, sturdy data mappings and neat analytics.

Essentially, a dashboard is the final outcome of a Siebel Analytics operation. Based on the metaphor of the instrument panel in a car, the Siebel Analytics version of a dashboard provides a user, mostly business managers, with the input necessary to “drive” the business. Siebel Analytics Interactive Dashboard refers to a web-based technology page on which real time or warehoused information (Reports) is embedded from various sources used in the business.


As seen in the above example of Interactive dashboards, business data is displayed infused with an array of user friendly views like Pie Charts, drill-downs, navigations, pivot table summaries, bar graphs, meters and gauges . These dashboards are often role-driven and customizable. They can be modeled to show historical, current and predictive data which leads to measure – based decision making. The biggest advantage of an Interactive dashboard is that it helps in decision making being more fact based and less intuitive.

Dashboard centric management is definitely going to be the next best thing in business analysis. The idea of viewing and analyzing the organizational facts and figures with the help of superb layouts in front of your computer screen is absolutely fantastic. In short, Dashboards are all about infusing ‘Intelligence’ in the business of the organization and a wonderful apparatus for any organization to possess.

Read More about Analyzing Analytics