Tuesday, May 7, 2013

The XML Logger

In my last post, I promised to try to bring us up to date on the current implementation of the logging framework, and specifically the XML Logger component of it.  The framework is initialized on the Application Start event making all these methods available from script.  From an XML Logging perspective, it can be initiated in one of two ways:

  • As a standard business service from within the Integration WF (or business service).  In this approach, just before the outbound WS or HTTP call, you would call the logRequest or logResponse methods of the framework business service, passing in at a minimum the property set about to be interfaced.  There are many other attributes of the payload record which can be optionally used which I won't go into detail over.  You can always add attributes to meet your needs and you don't need to populate any of them really.
  • As a Filter Service.  This is used for Web Services and is useful in that it can be turned on or off without modifying any existing Integration WFs.  On the Web Service admin views, for each web service operation that you want to log, just specify the Request/Response Filter Service as the Framework business service and the Request/Response Filter Method as logRequest/logResponse respectively.
  • Can be implemented to capture other payloads as needed, for instance before and after messages of the EAI Data Transformation Service
Ok, now for the nitty gritty.  What do the logRequest/logResponse methods do?  Both are similar and different only in that all interface logging records have a placeholder for a request and a response payload, which the two methods are differentiated to populate.  The main input to these methods is an XML payload.  At a high level, here is the algorithm:
  1. Navigate the property set until the first 'ListOf*' tag is found which is assumed to be the beginning of the Integration Object data.
  2. Call a method to Parse the remaining child data to correctly name and categorize the interface based on the IO Type and iddentify the record id and unique identifier attributes.  This allows for optional scripting to tailor the logging service to your client's unique needs
  3. Call the logInterface method which:
    1. Checks if in an EAI Transaction.  If so, add the payload to an array, otherwise continue (This is currently only implemented to support outbound interfaces when using the Filter service implementation)
    2. Creates a session record if one does not already exist (Inbound interfaces executed by an EAI OM typically)
    3. Deal with anonymous logins (when used on an inbound interface the request method will be executed under the Anonymous login but the response method with be performed by the interface user id)
    4. Creates a payload record to store the attributes extracted from the payload parsing
    5. Split the payload into chunks no larger than the BLOB length and create detail records for each chunk
First the PreInvoke method.  This mostly speaks for itself but since we may want to save processing overhead the calling of the parseInterface method is parameterized and controlled by which method is actually invoked.

function Service_PreInvokeMethod (MethodName, Inputs, Outputs){
  var retValue      = CancelOperation;

  switch(MethodName) {
  case "logInterface":     
    var key = logInterface(Inputs, Outputs);
    Outputs.SetProperty("key", key)
    break;
  case "logParseRequest":     
    logRequest(Inputs, Outputs, true, "");
    break;
  case "logParseResponse":     
    logResponse(Inputs, Outputs, true, "");
    break;
  case "logRequest":     
    logRequest(Inputs, Outputs, false, "");
    break;
  case "logResponse":     
    logResponse(Inputs, Outputs, false, "");
    break;
  case "logTransformRequest":     
    logRequest(Inputs, Outputs, false, "Transform");
    break;
  case "logTransformResponse":     
    logResponse(Inputs, Outputs, false, "Transform");
    break;
  }
  return (retValue);
}
Next the logRequest and logResponse Methods.  Like I said they are very similar except for which field the payload is passed to.  Also the logResponse method has some additional logic for parsing SOAP faults.

function logRequest(Inputs, Outputs, parse, mode) {
/* ***********************************
Purpose: Log interface request from/to an external interface in the session log
Usage: In Web Service definition, operations applet, set the Request Filter BS to 'PPT Utilities'
  and the method to logRequest.  Clear the cache
Arguments: 1 - SoapMessage will implicily be passed as a child property set
**************************************** */
try {
  var soapEnv, soapBody, divePS, direction, progress;
  var bodyType="";
  var msgType="";
  var parseResults = new Object();
  var key = TimeStamp("DateTimeMilli");
  var max = 3;
  var dives = 0;

  if(Inputs.GetChildCount() > 0) {
    // Get the SOAP envelope from the SOAP hierarchy.  If payload is passed as an input property set, skip down an extra level
    soapEnv = Inputs.GetChild(0);        //Like env:Envelope

    //Minimize processing if payloads/logging information will not be stored
    if (gHoldBufferDump == true || gsTraceIntfaceReqResp == "TRUE") {
      //if called from EAI Data Transformation Engine and user logging level is 5 capture passing specific props
      if (mode=="Transform" && (gHoldBufferDump == true || gCurrentLogLvl >= 5)) {
        parseResults.recId = soapEnv.GetChild(0).GetChild(0).GetProperty("Id");
        parseResults.recBC = soapEnv.GetChild(0).GetChild(0).GetType();
        msgType = Inputs.GetProperty("MapName");
        parseResults.funcName = Inputs.GetProperty("MapName");
        direction = "EAI Transform";
      } else {
        try { // Try to process the message to get functional data 
          if (soapEnv.GetType().toUpperCase().indexOf("ENVELOPE") <0 ) soapEnv = soapEnv.GetChild(0);
          direction = Inputs.GetProperty("WebServiceType")+" "+Inputs.GetProperty("Direction");
          for (var i=0; i < soapEnv.GetChildCount(); i++) {   
            bodyType = soapEnv.GetChild(i).GetType();    //Like env:Body
            if (bodyType.toUpperCase() == "BODY" || bodyType.substr(bodyType.indexOf(":")+1).toUpperCase() == "BODY") {
              soapBody = soapEnv.GetChild(i);
              for (var j=0; j < soapBody.GetChildCount(); j++) {   
                msgType = soapBody.GetChild(j).GetType();  //Full Port name of the WS 
  
                //Parse to check for faults and create a text string to be used in the key
                if (msgType.indexOf(":") >= 0)  msgType = msgType.substr(msgType.indexOf(":")+1); //strip namespace
                if (msgType.indexOf(" ") >= 0)  msgType = msgType.substr(0, msgType.indexOf(" ")); //strip namespace declaration
                if (msgType.indexOf("_") >= 0)  msgType = msgType.substr(0, msgType.lastIndexOf("_"));//strip port operation
  
                //if true, attempt to find Row Id in payload to stamp on log record so log can be linked to Siebel record  
                if (parse == true) {
                  divePS = soapBody.GetChild(j); //.GetChild(0)
                  while (divePS.GetType().indexOf("ListOf") < 0 && dives <= max) {
                    if (divePS.GetChildCount() > 0) {
                      divePS = divePS.GetChild(0);
                      dives++;
                    } else dives = max + 1;
                  }
   
                  //If a ListOf... container is found, this is a SiebelMessage generated by Siebel. Otherwise parse the SOAP Body
                  if (divePS.GetType().indexOf("ListOf") >= 0) parseInterface(divePS, parseResults);
                  else parseInterface(soapBody, parseResults);
                }
              } 
        
              break;
            }
   }         
 } catch(e) {
          //If an error occurs while parsing, just try to write the message whole
        }
            
      } //SOAP Message scenario

      //If msgType is identified then insert a log for this payload
      if (msgType != "") {
        msgType = msgType.replace(/_spc/g, "").replace(/\s/g, "");
        key = msgType+"_"+key;

        TheApplication().SetProfileAttr("InterfaceKeyInbound", key);
        progress = logInterface(key, soapEnv, null, direction, parseResults.recId, parseResults.recBC, "Pending", msgType, parseResults.funcName, null, null, null, parseResults.ref1, null, null, parseResults.refField);
      }
    } else if (gsTraceIntfaceReqResp == "FALSE") {
      //Do Nothing
    } else { //Store payloads in case an error occurs
      //var holdPayload = ["Request", Inputs, parse, mode];
      gHoldReqResp.push(["Request", Inputs, parse, mode]);
      if (gHoldBufferMax > 0 && gHoldReqResp.length > gHoldBufferMax) gHoldReqResp.shift();
    }
  } // Inputs.GetChildCount()
  Outputs.InsertChildAt(soapEnv,0);
} catch(e) {
  RaiseError(e);
} finally {
  divePS = null;
  soapBody = null;
  soapEnv = null;
  parseResults =  null;
}

}

function logResponse(Inputs, Outputs, parse, mode) {
/* ***********************************
Purpose: Log interface response from/to an external interface in the session log
Usage: In Web Service definition, operations applet, set the Response Filter BS to 'PPT Utilities'
  and the method to logResponse.  Clear the cache
Arguments: 1 - SoapMessage will implicily be passed as a child property set
**************************************** */
try {
  var soapEnv, soapBody, divePS, direction, progress;
  var bodyType="";
  var msgType="";
  var parseResults = new Object();
  var fault=null;
  var key = TheApplication().GetProfileAttr("InterfaceKeyInbound");
  var max = 3;
  var dives = 0;
  var dump = false;

  if(Inputs.GetChildCount() > 0) {
    // Get the SOAP envelope from the SOAP hierarchy
    soapEnv = Inputs.GetChild(0);

    //Minimize processing if payloads/logging information will not be stored
    if (gHoldBufferDump == true || gsTraceIntfaceReqResp == "TRUE") {
      if (mode=="Transform" && (gHoldBufferDump == true || gCurrentLogLvl >= 5)) {
        dump = true;
        direction = "EAI Transform";
        if (soapEnv.GetChild(0).GetChild(0).PropertyExists("Id")) {
          parseResults.recId = soapEnv.GetChild(0).GetChild(0).GetProperty("Id");
          parseResults.recBC = soapEnv.GetChild(0).GetChild(0).GetType();
        }
      } else if (mode=="") {
        dump = true;
        try { // Try to process the message to get functional data
          direction = Inputs.GetProperty("WebServiceType")+" "+Inputs.GetProperty("Direction");
          //Soap Envelope Request may have a Header and a Body so loop to the Body
          for (var i=0; i < soapEnv.GetChildCount(); i++) {
            bodyType = soapEnv.GetChild(i).GetType();
            if (bodyType.toUpperCase() == "BODY" || bodyType.substr(bodyType.indexOf(":")+1).toUpperCase() == "BODY") {
              soapBody = soapEnv.GetChild(i);
  
              //Soap Body typically has a container for the Message
              for (var j=0; j < soapBody.GetChildCount(); j++) {   
                msgType = soapBody.GetChild(j).GetType();
   
                //Parse to check for faults and create a text string to be used in the key
                if (msgType.indexOf(":") >= 0)  msgType = msgType.substr(msgType.indexOf(":")+1)
                if (msgType.indexOf(" ") >= 0)  msgType = msgType.substr(0, msgType.indexOf(" "))
                if (msgType.indexOf("_") >= 0)  msgType = msgType.substr(0, msgType.lastIndexOf("_"))
  
                if (msgType.toUpperCase() == "FAULT") fault = soapBody.GetChild(j).GetProperty("faultstring");
                else if (parse == true) { 
                  //if true, attempt to find Row Id in payload to stamp on log record so log can be linked to Siebel record  
                  divePS = soapBody.GetChild(j); //focus on the Message level
                  while (divePS.GetType().indexOf("ListOf") < 0 && dives <= max) {
                    if (divePS.GetChildCount() > 0) {
                      divePS = divePS.GetChild(0);
                      dives++;
                    } else dives = max + 1;
                  }
   
                  //If a ListOf... container is found, this is a SiebelMessage generated by Siebel. Otherwise parse the child of the Body
                  if (divePS.GetType().indexOf("ListOf") >= 0) parseInterface(divePS, parseResults);
                  else parseInterface(soapBody, parseResults);
                }
              }
              break;
            }
          }
        } catch(e) {
          //If an error occurs while parsing, just try to write the message whole
        }
      }
 
      if (key == "") {
        key = TimeStamp("DateTimeMilli");
        if (msgType != "") {
          msgType = msgType.replace(/_spc/g, "");
          key = msgType+"_"+key;
        }
      }
         
      if (dump == true) {
        if (fault != null) {
          logInterface(key, null, soapEnv, null, parseResults.recId, parseResults.recBC, "error", null, null, fault);
        } else {
          var recId = (parseResults.recId != "" ? parseResults.recId : null);
          var recBC = (parseResults.recBC != "" ? parseResults.recBC : null);
          var ref1 = (parseResults.ref1 != "" ? parseResults.ref1 : null);
          var funcName = (parseResults.funcName != "" ? parseResults.funcName : null);
          progress = logInterface(key, null, soapEnv, direction, recId, recBC, "Complete", msgType, funcName, null, null, null, ref1);
        }
      }
    } else if (gsTraceIntfaceReqResp == "FALSE") {
      //Do Nothing
    } else { //Store payloads in case an error occurs
      gHoldReqResp.push(["Response", Inputs, parse, mode]);
      if (gHoldBufferMax > 0 && gHoldReqResp.length > gHoldBufferMax) gHoldReqResp.shift();
    }
  } // Inputs.GetChildCount()
  Outputs.InsertChildAt(soapEnv,0);
} catch(e) {
  RaiseError(e);
} finally {
  divePS = null;
  soapBody = null;
  soapEnv = null;
  parseResults =  null;
  TheApplication().SetProfileAttr("InterfaceKeyInbound", "");
}
}

The parseInterface method passes the ListOf container of the Integration Object to a switch statement where each BC type can be evaluated.  This is useful only when using the Filter Service triggering mechanism otherwise the Record Id and Interface Name attributes can just be explicitly passed as parameters.  A case section should be created for each Integration Object being processed. This function really needs to be manually manipulated for every implementation and integration point to explicitly specify how to find the record id for a particular integration.

function parseInterface(ListOfIC, oReturn) {    //Input ListOfIC is a ListOf...
//Called from logRequest and logResponse to parse a message and get the row id or reference ids of different
//objects
try {
  if (ListOfIC.GetChildCount() > 0) {
    var IC = ListOfIC.GetChild(0);   //Integration Component Instance
    var icNameSpace = "";
    var intCompType = IC.GetType();
    var propName;
    var stop = false;
    var childIC;
    var childFlds;
      
    if (intCompType.indexOf(":") >= 0) {
      icNameSpace = intCompType.substr(0, intCompType.indexOf(":")+1);
      intCompType = intCompType.substr(intCompType.indexOf(":")+1);
    }

    //For these types, dive an additional level
    switch(intCompType) {
    case "ATPCheckInterfaceRequestOrders":
      IC = IC.GetChild(0);
      intCompType = IC.GetType();
      if (intCompType.indexOf(":") >= 0) {
        icNameSpace = intCompType.substr(0, intCompType.indexOf(":")+1);
        intCompType = intCompType.substr(intCompType.indexOf(":")+1);
      }
      break;
    }
 
    for (var flds = 0; flds < IC.GetChildCount(); flds++) { //Loop through Fields
      propName = IC.GetChild(flds).GetType();
      switch (intCompType) {
      case "Quote":
      case "SWIQuote":
        oReturn.recBC = "Quote";
        if (propName == icNameSpace+"Id") {
          oReturn.recId = IC.GetChild(flds).GetValue();
          stop = true;
        }
        break;

      case "ProductIntegration":
        oReturn.recBC = "Internal Product";
        if (propName == icNameSpace+"ListOfProductDefinition") {
          childIC = IC.GetChild(flds).GetChild(0);
          for (childFlds = 0; childFlds < childIC.GetChildCount(); childFlds++) { //Loop through Fields
            propName = childIC.GetChild(childFlds).GetType();
            if (propName == icNameSpace+"Id" || propName == icNameSpace+"ProductId") {
              oReturn.recId = childIC.GetChild(childFlds).GetValue();
              stop = true;
            }
            if (stop) break;                              
          }
        }
        break;
     
      default:
        if (propName.indexOf("Id") >= 0) {
          oReturn.recId = IC.GetChild(flds).GetValue();
          stop = true;
        }
        oReturn.recBC = intCompType;
        stop = true;
      }
      if (stop) break;                              
    }
  }
} catch(e) {
  RaiseError(e);
} finally {
  childIC = null;
  IC = null;
}
}

The logInterface method is called by both logRequest and logResponse to manage the session records for inbound interfaces and create the actual payload record.  I will have to go into more detail about the Anonymous login processing at some other time.  Suffice to say this works under a variety of web service setups.

function logInterface() {
if (gHoldBufferDump == true || gsTraceIntfaceReqResp == "TRUE") {
  try {
    var key, dir, recId, recBC, status, srcObj, name, logText, userText, retCode, ref1, ref2, ref3, refField, findResponseMethod;
    var request:PropertySet, response:PropertySet;
    var progress = "";

    //if attributes passed as an input property set, set variables from them
    if (typeof(arguments[0]) == "object") {
      progress = progress+"\n"+"REF1: typeof(arguments[0]) == object";
      var Inputs:PropertySet = arguments[0];
      name = Inputs.GetProperty("FunctionalName");
      key = Inputs.GetProperty("Key");
   
      for (var i=0;i < Inputs.GetChildCount();i++) {
        if (Inputs.GetChild(i).GetType() == "Request") request = Inputs.GetChild(0);
        if (Inputs.GetChild(i).GetType() == "Response") response = Inputs.GetChild(0);
      }
      dir = Inputs.GetProperty("Direction");
      recId = Inputs.GetProperty("LinkId");
      recBC = Inputs.GetProperty("LinkBC");
      status = Inputs.GetProperty("Status");
      srcObj = Inputs.GetProperty("SourceObject");
      logText = Inputs.GetProperty("LogText");
      userText = Inputs.GetProperty("UserText");
      retCode = Inputs.GetProperty("ReturnCode");
      ref1 = Inputs.GetProperty("Reference1");
      ref2 = Inputs.GetProperty("Reference2");
      ref3 = Inputs.GetProperty("Reference3");
      refField = Inputs.GetProperty("RefField");
    } else {
      progress = progress+"\n"+"REF1: else typeof(arguments[0])";
      key = arguments[0];
      request = arguments[1];
      response = arguments[2];
      dir = arguments[3];
      recId = arguments[4];
      recBC = arguments[5];
      status = arguments[6];
      srcObj = arguments[7];
      name = arguments[8];
      logText = arguments[9];
      userText = arguments[10];
      retCode = arguments[11];
      ref1 = arguments[12];
      ref2 = arguments[13];
      ref3 = arguments[14];
      refField = arguments[15];
    }
  
    //When called though WF as Payload logger, generate the key if not provided
    if (key == "" || key == undefined || key == null) {
      if (name == "" || name == undefined || name == null) name = "None";
      key = name.replace(/_spc/g, "").replace(/\s/g, "")+TimeStamp();
    }

    var found:Boolean = false;
    var sessionId:String = "";
    var guestSessionId:String = "";
    var guestSessionFound = false;
    var createSession = false;
    var firstMessage = false;
    var boSessionFlat;
    var bcSessionXMLFlat;
    var useGuestSession = (response != null && gsMergeGuestSessions=="TRUE" && gsGuest != "" ? true : false);
    var boSession:BusObject = TheApplication().GetBusObject("PPT User Session");
    var bcSession:BusComp;
 
    if (gHoldBuffer == true) {
      progress = progress+"\n"+"REF2: gHoldBuffer == true";
      //If in a EAI Txn, store all payloads in an array so they can be written after the commit or rollback
      gHoldPayloads[gHoldPayloads.length] = arguments;
    } else {
      progress = progress+"\n"+"REF2: gsTraceIntfaceReqResp == TRUE & key == "+key;
      //If an interface is being logged for the first time, need to instantiate the session
      if (gSessionId == "") {
        progress = progress+"\n"+"REF3: gSessionId == ''";
        //If Guest connections are not used, a simplified session management can be used 
        if (gsGuest == "" && key != "") {
          progress = progress+"\n"+"REF4: gsGuest == '' & key == "+key;
          bcSession = boSession.GetBusComp("PPT User Session");
          with (bcSession) {
            NewRecord(NewBefore);
            SetFieldValue("Employee Id",TheApplication().LoginId());
            SetFieldValue("Session Stamp",gsLogSession);
            WriteRecord();
            gSessionId = GetFieldValue("Id");
          }
          createSession = false; //skip logic below to create/merge a guest session
          firstMessage = true; //will always insert the input message rather than querying to update
          //Reset the variable to check whether to merge the guest session.  This allows a log buffer dump
          gsMergeGuestSessions = "FALSE";
        } else {
          progress = progress+"\n"+"REF4: else: gsGuest == '' & key == "+key;
          createSession = true;
  
          //confirm that current session has not been created yet
          bcSession = boSession.GetBusComp("PPT User Session");
          with (bcSession) {
            ClearToQuery();
            SetSearchSpec("Session Stamp", gsLogSession);
            ExecuteQuery(ForwardOnly);
            found = FirstRecord();
 
            if (found == true) {
              gSessionId = GetFieldValue("Id");
              createSession = false;
            } else {
              firstMessage = true;
            }
          }
        }
      }
 
      if (createSession == true || useGuestSession == true) {
        progress = progress+"\n"+"REF5: createSession == true || useGuestSession == true";
        bcSession = boSession.GetBusComp("PPT User Session");
 
        //Because EAI logins can trigger logging from the anonymous login, the response logging will trigger
        //from a different session. Query for the most recent corresponding request log and update it
        if (useGuestSession) {
          progress = progress+"\n"+"REF6: useGuestSession";
          boSessionFlat = TheApplication().GetBusObject("PPT User Session Flat");
          bcSessionXMLFlat = boSessionFlat.GetBusComp("PPT User Session XML");
          bcSessionXMLFlat.ActivateField("Parent Id");
          bcSessionXMLFlat.ClearToQuery();
          if (typeof(srcObj) != "undefined" && srcObj != null) bcSessionXMLFlat.SetSearchSpec("Source Object", srcObj);
          if (typeof(ref1) != "undefined" && ref1 != null) bcSessionXMLFlat.SetSearchSpec("Reference Id 1", ref1);
          bcSessionXMLFlat.SetSearchSpec("Response", "IS NULL");
          bcSessionXMLFlat.SetSearchSpec("Employee Login", gsGuest);
          bcSessionXMLFlat.SetSortSpec("Created (DESC)");
          bcSessionXMLFlat.ExecuteQuery(ForwardBackward);
          guestSessionFound = bcSessionXMLFlat.FirstRecord();
    
          if (guestSessionFound == true) guestSessionId = bcSessionXMLFlat.GetFieldValue("Parent Id");
        }
    
        if (guestSessionFound == false && createSession == true) {
          progress = progress+"\n"+"REF7: guestSessionFound == false & createSession == true";
          //Anonymous login session not found and there is no current session.  Create a new one
          with (bcSession) {
            NewRecord(NewBefore);
            SetFieldValue("Employee Id",TheApplication().LoginId());
            SetFieldValue("Session Stamp",gsLogSession);
            WriteRecord();
            gSessionId = GetFieldValue("Id");
          }
        } else if (guestSessionFound == true && createSession == false) {
          progress = progress+"\n"+"REF7: guestSessionFound == true & createSession == false";
          //Anonymous login session found and there is a current session. 
          //Link child records to the parent session for the Interface User and delete the guest session (faster than a merge)
          while (guestSessionFound) {
            bcSessionXMLFlat.SetFieldValue("Parent Id",gSessionId);
            bcSessionXMLFlat.WriteRecord();
            guestSessionFound = bcSessionXMLFlat.NextRecord();
          }
          with (bcSession) {
            ClearToQuery();
            SetSearchSpec("Id", guestSessionId);
            SetSortSpec("");
            ExecuteQuery(ForwardOnly);
            guestSessionFound = FirstRecord();
     
            if (guestSessionFound == true) DeleteRecord();
            ClearToQuery();
            SetSortSpec("");
            SetSearchSpec("Id", gSessionId);
            ExecuteQuery(ForwardOnly);
          }
        } else if (guestSessionFound == true && createSession == true) {
          progress = progress+"\n"+"REF7: guestSessionFound == true & createSession == true";
          //Anonymous login session found and there is no current session.  Update the guest session to EAI values
          with (bcSession) {
            ActivateField("Employee Id");
            ActivateField("Session Stamp");
            ClearToQuery();
            SetSearchSpec("Id", guestSessionId);
            SetSortSpec("");
            ExecuteQuery(ForwardBackward);
            found = FirstRecord();
     
            if (found == true) {
              SetFieldValue("Employee Id",TheApplication().LoginId());
              SetFieldValue("Session Stamp",gsLogSession);
              WriteRecord();
            }
          }
        } else {
          progress = progress+"\n"+"REF7: Anonymous login session not found and there is a current session.  Do Nothing";
          //Anonymous login session not found and there is a current session.  Do Nothing
        }
 
        //Reset the variable to check whether to merge the guest session.  This allows a log buffer dump
        gsMergeGuestSessions = "FALSE";
      }
      var bcSessionXML = boSession.GetBusComp("PPT User Session XML");
      var bcSessionXMLDtl = boSession.GetBusComp("PPT User Session XML Detail");
 
      with (bcSessionXML) {
        if (firstMessage) {
          progress = progress+"\n"+"REF8: firstMessage";
          //This is an insert so no query needed to update an existing record
          found = false;
          findResponseMethod = "Do Not Search";
        } else if (useGuestSession && guestSessionFound) {
          progress = progress+"\n"+"REF8: useGuestSession & guestSessionFound";
          //If this is the first update after a guest session is used, the key will not match but there should only be one XML record
          ClearToQuery();
          SetSearchSpec("Created By Login", gsGuest);
          SetSearchSpec("Parent Id", gSessionId);
          SetSearchSpec("Request", "IS NOT NULL");
          SetSearchSpec("Response", "IS NULL");
          ExecuteQuery(ForwardBackward);
          found = FirstRecord();
          findResponseMethod = "Session/Guest Login: "+gSessionId+"/"+gsGuest;
        } else if ((typeof(response) != "undefined" && response != null) ||
          (typeof(ref1) != "undefined" && ref1 != null && typeof(request) != "undefined" && request != null && gsReplaceSyncResponseWAsyncRequest == "TRUE")) {
          progress = progress+"\n"+"REF8: normal response update OR inbound request to a previously sent asynchronous request";
          //This is a normal response update to an existing message record or 
          //it is an inbound request to a previously sent asynchronous request with a matching Reference Id AND we want to log the Async request as the response
          ClearToQuery();
          SetSearchSpec("Parent Id", gSessionId);
 
          //If this is an Inbound request and Ref1 is provided, lookup by Ref1 
          if (typeof(ref1) != "undefined" && ref1 != null && typeof(request) != "undefined" && request != null) {
            SetSearchSpec("Reference Id 1", ref1.substring(0, 100));
            findResponseMethod = "Reference 1: "+ref1.substring(0, 100);
          } else {
            SetSearchSpec("Name", key);
            findResponseMethod = "Key: "+key;
          }
          SetSortSpec("");
          ExecuteQuery(ForwardBackward);
          found = FirstRecord();
        }
    
        //This is a normal request or an existing record could not be found for some reason
        if (found == false) {
          progress = progress+"\n"+"REF9: This is a normal request or an existing record could not be found for some reason ("+findResponseMethod+")";
          NewRecord(NewBefore);
          SetFieldValue("Name",key);
          SetFieldValue("Parent Id",gSessionId);
        }
    
        if (useGuestSession == true) {
          progress = progress+"\n"+"REF10: Set Name Key: "+key.substring(0, 100);
          SetFieldValue("Name", key.substring(0, 100));
        }
 
        //Since payloads can be any size, very large ones greater than CLOB size 131k need to be split with the text being 
        //put into multiple records
        if (typeof(request) != "undefined" && request != null) {
          progress = progress+"\n"+"REF11: Set Request";
          SetFieldValue("Request Time", TimeStamp("DateTimeFormatted"));
          splitPayload(bcSessionXML, bcSessionXMLDtl, "Request", request, 131072)
        }
        if (typeof(response) != "undefined" && response != null) {
          progress = progress+"\n"+"REF12: Set Response";
          SetFieldValue("Response Time", TimeStamp("DateTimeFormatted"));
          splitPayload(bcSessionXML, bcSessionXMLDtl, "Response", response, 131072)
        }
  
        if (typeof(dir) != "undefined" && dir != null && dir != "")   SetFieldValue("Direction", dir.substring(0, 30));
        if (typeof(recId) != "undefined" && recId != null && recId != "") SetFieldValue("Record Id", recId.substring(0, 15));
        if (typeof(recBC) != "undefined" && recBC != null && recBC != "") SetFieldValue("Record BC", recBC.substring(0, 50));
        if (typeof(status) != "undefined" && status != null && status !="") SetFieldValue("Status", status.substring(0, 30));
        if (typeof(srcObj)!="undefined" && srcObj != null && srcObj !="") SetFieldValue("Source Object", srcObj.substring(0, 50));
        if (typeof(name) != "undefined" && name != null && name != "")  SetFieldValue("Functional Name", name.substring(0, 50));
        if (typeof(logText)!= "undefined" && logText != null && logText !="") SetFieldValue("Log Text", logText.substring(0, 131072));
        if (typeof(userText)!= "undefined" && userText != null && userText !="") SetFieldValue("User Text", userText.substring(0, 255));
        if (typeof(retCode) != "undefined" && retCode != null && retCode != "")  SetFieldValue("Return Code", retCode.substring(0, 30));
        if (typeof(refField) != "undefined" && refField != null && refField!="") {
          SetFieldValue(refField, ref1);
          SetFieldValue("Reference Id 2", refField);
        } else {
          if (typeof(ref1) != "undefined" && ref1 != null)   SetFieldValue("Reference Id 1", ref1.substring(0, 100));
          if (typeof(ref2) != "undefined" && ref2 != null)   SetFieldValue("Reference Id 2", ref2.substring(0, 100));
        }
        if (typeof(ref3) != "undefined" && ref3 != null)   SetFieldValue("Reference Id 3", ref3.substring(0, 100));
        progress = progress+"\n"+"REF13: PreWrite";
        WriteRecord();
      }
      if (gsLogMode == "FILE") {
        progress = progress+"\n"+"REF14: gsLogMode == FILE";
        if (typeof(request) != "undefined" && request != null)  PropSetToFile(key+"_Request", request);
        if (typeof(response) != "undefined" && response != null) PropSetToFile(key+"_Response", response);
      }
    }
  } catch(e) {
    RaiseError(e, progress);
  } finally {
    request = null;
    response = null;
    Inputs = null;
    bcSessionXML = null;
    bcSessionXMLFlat = null;
    boSessionFlat = null;
    bcSession = null;
    boSession = null;
    TheApplication().SetProfileAttr("C1 Session Id", gSessionId);
  }
  return(progress);
}
}

The trimPS method is actually deprecated in my service but I include it in case it is useful to anyone.  I basically just takes a property set, converts it to text, then sets a specified field on an instantiated BC passed as an input to the converted text of the property set.  The assumption in using this method is that there is a limit to how large the stored payload can be.

function trimPS(ps, fieldName, bc, maxLength){
try {
  var psIn      = TheApplication().NewPropertySet();
  var psOut     = TheApplication().NewPropertySet();
  var bsService = TheApplication().GetService ("XML Converter (Data in Child PropSet)");
  psIn.SetProperty("EscapeNames", "False");
  var text:String = "";

  psIn.AddChild(ps);
  bsService.InvokeMethod("PropSetToXML", psIn, psOut);
  bc.SetFieldValue(fieldName, ToString(psOut).substring(0, maxLength));
  bc.SetFieldValue(fieldName+" Time", TimeStamp("DateTimeFormatted"));
  text = bc.GetFieldValue(fieldName);
  text = text.lTrim();
  if (text.substring(0, 14) == "PropertySet [ ") text = text.substring(14);
  text = text.rTrim("]");
  bc.SetFieldValue(fieldName,text);
} catch (e) {
  throw(e);
} finally { 
  psOut = null;
  psIn = null;
  bsService = null;
}
}

The splitPayload method is what replace the trimPS method.  It is no longer limited in being able store only a certain character length of payload as this method splits the payload into chunks of a specified size and inserts records into the instantiated BC passed as an input.

function splitPayload(parentbc, detailbc, fieldName, ps, maxLength) {
try {
  var psIn      = TheApplication().NewPropertySet();
  var psOut     = TheApplication().NewPropertySet();
  var bsService = TheApplication().GetService ("XML Converter (Data in Child PropSet)");
  psIn.SetProperty("EscapeNames", "False");
  var text:String = "";
  var stripPS = false;
  var textPS = "";
  
  psIn.AddChild(ps);
  bsService.InvokeMethod("PropSetToXML", psIn, psOut);
  textPS = ToString(psOut);

  if (textPS.length > maxLength) {
    while (textPS.length > 0) {
      detailbc.NewRecord(NewAfter);
      detailbc.SetFieldValue("Field", fieldName);
      detailbc.SetFieldValue("Log Text", textPS.substring(0, maxLength));
 
      //Service adds a Prefix that needs to be removed
      text = detailbc.GetFieldValue("Log Text");
      text = text.lTrim();
      if (text.substring(0, 14) == "PropertySet [ ") {
        stripPS = true;
        text = text.substring(14);
      }
 
      textPS = textPS.substring(maxLength, textPS.length); 
 
      //If the Text is broken up across multiple records, need to remove the trailing ] from the last record
      if (stripPS && textPS.length < maxLength) {
        text = text.rTrim("]");
      }
      detailbc.SetFieldValue("Log Text",text);
      detailbc.WriteRecord();
    }
  } else {
    parentbc.SetFieldValue(fieldName, textPS);
    text = parentbc.GetFieldValue(fieldName);
    text = text.lTrim();
    if (text.substring(0, 14) == "PropertySet [ ") {
      stripPS = true;
      text = text.substring(14).rTrim("]");
      parentbc.SetFieldValue(fieldName, text);
    }
//  parentbc.WriteRecord();
  }
} catch (e) {
  RaiseError(e);
} finally { 
  psOut = null;
  psIn = null;
  bsService = null;
}
}

Next: Viewing the Payload

State of Logging - An Introduction

I was reading Jason Le's recent post about XML Logging and I realized the problem statement he was describing was intimately familiar to me because it is something I have been dealing with over the years and have evolved a solution for it.  I say evolved because it has gone though so many iterations by this point I am not sure where to begin.  So let me start with a verbal explanation, and then I will get into the details.

I built an eScript Logging Service back in the day to help with tracing processing through custom script.  This was written out to a text file in the file system.  I don't recall the sequence of events after that but at some point I added an XML logging utility to this so that the XML files would be written to the file system as well. This was initially an explicit call to a business service within the integration WF to pass an XML Property Set  so it could be written out.  I then moved to a client which was using a business service to create custom SOAP headers so I modified the logging service so that it could be called as a custom filtering service itself.  This allowed the exact payload to be captured as it was leaving/returning to the Siebel application.

At some point I ended up on a client that had UNIX Application servers, needed me to do production support post go-live AND was very restrictive about granting access to the production file system.  Rather than deal with the UNIX and access issues, I opted to modify the service so it could operate in two modes.  When using a thick client, it would continue to write all information out to the local file system as defined by the SIEBEL_LOG_DIR environment variable.  But I allowed thin client users to output logging data to a new set of custom tables.  Basically when a user logs in, a session record is created.  All escript logging data is written out to a 1:M detail table in buffered chunks to minimize performance hits.  XML Logging does the same (though in a slightly different way).  Each detail record has the opportunity to store a row id which allows a session to be linked to a particular record that was logged, like an Order Id.  This allows for audit views to be created which show all the user sessions with logging information tied to a particular record.

The other problem I needed to solve was that when an error occurs, Siebel rolls back transactions.  Therefore all interfaces use the EAI Transaction Service and I have modified this service to set a flag so that when in a transaction, all XML payloads are stored in memory until the transaction is either aborted or committed, at which point all the messages are written to the DB in the order they were executed.

To view logging information, whether XML or escript, requires the many detail records to be reassembled into a viewable format.  There is a method of the service that basically just queries all the detail records for a particular session or XML Payload and concatenates the text into a single value.  This method can be invoked from a calculated field and the calculated field exposed on a form applet.  Because form applet fields are not the best way to see the data, I typically copy the data out of the field and into a text editor of choice.  For instance I use Notepad++  with a Pretty Print add on to view XML payloads.

It will take several posts to go over the implementation in detail, but hopefully this whets people's appetites

Next: The XML Logger

Wednesday, December 12, 2012

Miscellaneous SQL Repository Research

I frequently use SQL to track down potential issues in the repository.  Here are a couple of statements that help solve common problems:

Fields Inactivated in the BC but Active in the Integration Object

select io.name int_obj, ic.name int_comp, icf.NAME int_field from siebel.S_INT_FIELD icf, siebel.S_INT_COMP ic, siebel.S_INT_OBJ io, siebel.s_repository r, siebel.s_buscomp bc, siebel.s_field f
where icf.INT_COMP_ID = ic.row_id and ic.INT_OBJ_ID = io.row_id and io.repository_id = r.row_id and r.name = 'Siebel Repository'
and ic.EXT_NAME = bc.name and icf.EXT_NAME = f.name and f.buscomp_id = bc.row_id and bc.repository_id = r.row_id and f.inactive_flg = 'Y'
and io.inactive_flg = 'N' and ic.inactive_flg = 'N' and icf.inactive_flg = 'N' and f.last_upd > to_date('11/01/2012', 'MM/DD/YYYY');

 Identify potential causes of the Truncation or Null Fetch error

This can sometimes be caused by not compiling the Table object too.  The first query finds business component joined fields where the BC field length or data type does not match the column length.  The second does the same but only analyzes base table columns and adds a last updated parameter which could be added or removed from either.  The last updated date is useful because vanilla actually has numerous instances of this potential problem.  I say potential because any records returned need to be checked against the data.  The error will only occur if the data in the column is longer than the BC length specified.  The Third query is an example of how to check assuming for instance S_ORDER_ITEM.ATTRIB_40 is mapped to a DTYPE_ID BC field..

select bc.name Bus_comp, f.name field, f.join_name Join, j.DEST_TBL_NAME Tbl_name, f.col_name Col_name, f.type Field_Type, f.textlen Fld_Lgth, c.DATA_TYPE Col_type, c.LENGTH col_lgth
from siebel.s_field f, siebel.s_repository r, siebel.s_join j, siebel.s_buscomp bc, siebel.s_column c, siebel.s_table t
where bc.REPOSITORY_ID = r.row_id and r.name = 'Siebel Repository'
and bc.row_id = f.buscomp_id and j.name = f.join_name and j.buscomp_id = bc.row_id
and c.TBL_ID = t.row_id and t.NAME = j.DEST_TBL_NAME and c.name = f.col_name
and r.row_id = t.REPOSITORY_ID and r.row_id = f.REPOSITORY_ID and r.row_id = c.REPOSITORY_ID
and t.INACTIVE_FLG = 'N' and c.INACTIVE_FLG = 'N' and bc.INACTIVE_FLG = 'N' and f.INACTIVE_FLG = 'N'
and ((f.type = 'DTYPE_ID' and c.length > 15)
or (f.type = 'DTYPE_PHONE' and c.length > 40)
or (f.type = 'DTYPE_BOOL' and c.length > 1)
or (f.type = 'DTYPE_TEXT' and c.DATA_TYPE like 'Date%'))
order by j.DEST_TBL_NAME, f.col_name;

select bc.name Bus_comp, f.name field, t.NAME tbl_name, f.col_name Col_name, f.type Field_Type, f.textlen Fld_Lgth, c.DATA_TYPE Col_type, c.LENGTH col_lgth
from siebel.s_field f, siebel.s_repository r, siebel.s_buscomp bc, siebel.s_column c, siebel.s_table t
where bc.REPOSITORY_ID = r.row_id and r.name = 'Siebel Repository'
and bc.row_id = f.buscomp_id
and c.TBL_ID = t.row_id and t.NAME = bc.TABLE_NAME and c.name = f.col_name
and r.row_id = t.REPOSITORY_ID and r.row_id = f.REPOSITORY_ID and r.row_id = c.REPOSITORY_ID
and t.INACTIVE_FLG = 'N' and c.INACTIVE_FLG = 'N' and bc.INACTIVE_FLG = 'N' and f.INACTIVE_FLG = 'N'
and ((f.type = 'DTYPE_ID' and c.length > 15)
or (f.type = 'DTYPE_PHONE' and c.length > 40)
or (f.type = 'DTYPE_BOOL' and c.length > 1)
or (f.type = 'DTYPE_TEXT' and c.DATA_TYPE like 'Date%'))
and f.last_upd > to_date('06/01/2010', 'MM/DD/YYYY')
order by t.NAME, f.col_name;

select ATTRIB_40 from siebel.S_ORDER_ITEM_XM where length(ATTRIB_40) > 15;

WF Deployed but not activated:

select wpr.proc_name, wpr.version from siebel.S_WFR_PROC wpr, siebel.s_repository r
where r.row_id = wpr.REPOSITORY_ID and r.name = 'Siebel Repository' and wpr.STATUS_CD = 'COMPLETED'
and wpr.name like 'PPT%'
and exists (select 'x' from siebel.S_WFA_DPLOY_DEF wpd where wpd.NAME = wpr.proc_name and wpd.TYPE_CD = 'PROCESS' and wpd.DEPLOY_STATUS_CD = 'ACTIVE'
            and wpd.REPOSITORY_VERSION <> wpr.VERSION)
and wpr.version = (select max(wpr1.version) from siebel.S_WFR_PROC wpr1
                  where r.row_id = wpr1.REPOSITORY_ID and wpr1.STATUS_CD = 'COMPLETED' and wpr1.proc_name = wpr.proc_name)
order by wpr.proc_name;

BC Fields not configured as boolean that should be

select bc.name, f.name, f.type, f.CALCVAL from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository'
and (f.CALCVAL like '%,%Y%,%N%)' OR f.CALCVAL like '%,%N%,%Y%)')
and f.TYPE <> 'DTYPE_BOOL'
and f.inactive_flg = 'N' and bc.name like 'PPT%' and bc.inactive_flg = 'N'
order by bc.name, f.name;

select bc.name, f.name, f.type, f.CALCVAL from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository'
and f.TEXTLEN = 1
and f.TYPE <> 'DTYPE_BOOL'
and f.inactive_flg = 'N' and bc.name like 'PPT%' and bc.inactive_flg = 'N'
order by bc.name, f.name;

Applet controls where checkbox not configured correctly

select a.name, lc.name, lc.field_name, lc.HTML_TYPE, lc.HTML_ICON_MAP, f.type from siebel.s_list_column lc, siebel.s_list l, siebel.s_applet a, siebel.s_repository r, siebel.s_field f, siebel.s_buscomp bc
where lc.list_id = l.row_id and l.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and lc.inactive_flg = 'N' and a.inactive_flg = 'N'
and (lc.HTML_ICON_MAP is null or lc.HTML_TYPE <> 'CheckBox')
and f.name = lc.FIELD_NAME and f.BUSCOMP_ID = bc.row_id and bc.name = a.BUSCOMP_NAME and f.TYPE = 'DTYPE_BOOL' and a.name like 'PPT%';

Integration Object Maps that have changed since a prior release:

select release, Map_Name, change_level, Field_Upd, Field_User, Comp_Upd, Comp_User, Obj_Upd, Obj_user, Comp_Name, src_expr, dst_int_fld_name, Obj_Comments
from ( -- Object level
select o.x_release release, o.name Map_Name, '1-Object' change_level, null Field_Upd, null Field_User, null Comp_Upd, null Comp_User, o.last_upd Obj_Upd, ou.login Obj_user, null Comp_Name, null src_expr, null dst_int_fld_name, o.comments Obj_Comments
from siebel.S_INT_OBJMAP o, siebel.s_user ou
where o.last_upd_by = ou.row_id and o.last_upd >= to_date('04/23/2019', 'MM/DD/YYYY')
UNION ALL -- Component level
select o.x_release release, o.name Map_Name, '2-Component' change_level, null Field_Upd, null Field_User, c.last_upd Comp_Upd, cu.login Comp_User, null Obj_Upd, null Obj_user, c.name Comp_Name, c.SRC_SRCHSPEC src_expr, null dst_int_fld_name, o.comments Obj_Comments
from siebel.S_INT_OBJMAP o, siebel.S_INT_COMPMAP c, siebel.s_user cu, siebel.s_user ou
where c.int_obj_map_id = o.row_id and c.last_upd_by = cu.row_id and o.last_upd_by = ou.row_id --and o.x_release = '1.27.12'
and c.last_upd >= to_date('4/23/2019', 'MM/DD/YYYY')
UNION ALL -- Field level
select o.x_release release, o.name Map_Name, '3-Field' change_level, f.last_upd Field_Upd, fu.login Field_User, null Comp_Upd, null Comp_User, null Obj_Upd, null Obj_user, c.name Comp_Name, f.src_expr, f.dst_int_fld_name, o.comments Obj_Comments
from siebel.S_INT_OBJMAP o, siebel.S_INT_COMPMAP c, siebel.S_INT_FLDMAP f, siebel.s_user fu, siebel.s_user cu, siebel.s_user ou
where f.int_comp_map_id = c.row_id and c.int_obj_map_id = o.row_id and f.last_upd_by = fu.row_id and c.last_upd_by = cu.row_id and o.last_upd_by = ou.row_id --and o.x_release = '1.27.12'
and (f.last_upd >= to_date('4/23/2019', 'MM/DD/YYYY'))
) where release <> '1.27.13'
order by Map_Name, change_level;

Repository Search by SQL

Tools offers a Repository Search feature which is quite robust but it has some significant drawbacks:
  1. Tedious to specify only those items which might have the object name you are looking for
  2. Time consuming to do a full search or a search of certain objects with high record counts
  3. Does not search UI admin objects where a field might be referenced
There have been times when I wanted to inactivate objects in order to streamline the application and have had difficulty identifying all the references.  Or if you see an error in the Siebel log, it is sometimes hard to forensically determine its source.

I have written a series of SQL statements to search for the items directly against the server tools DB.  This SQL looks at both repository tables where a particular object type might be referenced as well as UI administrative objects.  It is not guaranteed to be 100% comprehensive and should be modified to the particular needs of how a client has used the application.  For instance, if Variable Maps are not modified from vanilla, it is probably not necessary to query the tables that store them.

Fields
Applets

SQL Applet Search

Here is the SQL to find Applet References.  You will notice there are two queries of Toggle Applets.  One rolls an applet all the way up to the view it appears in while the other shows toggle applets that do not appear in any view.  Therefore, in a toggle sequence, there may be one applet that appears twice.  The wc or wildcard bind variable adds a trailing wildcard

var :applet = "Quote Form Applet";     -- Applet Name
var :wc = "N";                               

select attr_type, screen, viewname, obj_name, attr_name, Applet_Mode from (
select 'View web template item' attr_type, s.name screen, v.name viewname, '' obj_name, vwti.APPLET_NAME attr_name, vwti.APPLET_MODE_CD Applet_Mode from siebel.s_screen s, siebel.S_SCREEN_VIEW sv, siebel.S_VIEW_WTMPL_IT vwti, siebel.S_VIEW_WEB_TMPL vwt, siebel.s_view v, siebel.s_repository r
where vwti.APPLET_NAME like :applet||decode(:wc,'Y','%','') and vwti.VIEW_WEB_TMPL_ID = vwt.row_id and vwt.VIEW_ID = v.row_id and v.name = sv.VIEW_NAME and sv.SCREEN_ID = s.row_id and s.repository_id = r.row_id
and v.repository_id = r.row_id and r.name = 'Siebel Repository' and vwti.inactive_flg = 'N' and vwt.inactive_flg = 'N' and sv.inactive_flg = 'N' and v.inactive_flg = 'N' and s.inactive_flg = 'N'

union all

select 'Form Applet Pick Applet' attr_type, '' screen, '' viewname, a.name obj_name, c.PICK_APPLET_NAME attr_name, '' Applet_Mode from siebel.s_control c, siebel.s_applet a, siebel.s_repository r
where c.PICK_APPLET_NAME like :applet||decode(:wc,'Y','%','') and c.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and c.inactive_flg = 'N' and a.inactive_flg = 'N'

union all

select 'Form Applet MVG Applet' attr_type, '' screen, '' viewname, a.name obj_name, c.MVG_APPLET_NAME attr_name, '' Applet_Mode from siebel.s_control c, siebel.s_applet a, siebel.s_repository r
where c.MVG_APPLET_NAME like :applet||decode(:wc,'Y','%','') and c.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and c.inactive_flg = 'N' and a.inactive_flg = 'N'

union all

select 'List Applet Pick Applet' attr_type, '' screen, '' viewname, a.name obj_name, lc.PICK_APPLET_NAME attr_name, '' Applet_Mode from siebel.s_list_column lc, siebel.s_list l, siebel.s_applet a, siebel.s_repository r
where lc.PICK_APPLET_NAME like :applet||decode(:wc,'Y','%','') and lc.list_id = l.row_id and l.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and lc.inactive_flg = 'N' and a.inactive_flg = 'N'

union all

select 'List Applet MVG Applet' attr_type, '' screen, '' viewname, a.name obj_name, lc.MVG_APPLET_NAME attr_name, '' Applet_Mode from siebel.s_list_column lc, siebel.s_list l, siebel.s_applet a, siebel.s_repository r
where lc.MVG_APPLET_NAME like :applet||decode(:wc,'Y','%','') and lc.list_id = l.row_id and l.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and lc.inactive_flg = 'N' and a.inactive_flg = 'N'

union all

select 'Associate Applet' attr_type, '' screen, '' viewname, a.name obj_name, a.ASSOC_APPLET_NAME attr_name, '' Applet_Mode from siebel.s_applet a, siebel.s_repository r
where a.ASSOC_APPLET_NAME like :applet||decode(:wc,'Y','%','') and a.repository_id = r.row_id and r.name = 'Siebel Repository' and a.inactive_flg = 'N'
union all
select 'Applet Toggle' attr_type, '' screen, '' viewname, a.name obj_name, t.APPLET_NAME attr_name, '' Applet_Mode from siebel.S_APPLET_TOGGLE t, siebel.s_applet a, siebel.s_repository r
where t.APPLET_NAME like :applet||decode(:wc,'Y','%','') and t.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and t.inactive_flg = 'N' and a.inactive_flg = 'N'

union all

select 'Applet Toggle' attr_type, s.name screen, v.name viewname, a.name obj_name, t.APPLET_NAME attr_name, vwti.APPLET_MODE_CD Applet_Mode from siebel.s_screen s, siebel.S_SCREEN_VIEW sv, siebel.S_VIEW_WTMPL_IT vwti, siebel.S_VIEW_WEB_TMPL vwt, siebel.s_view v, siebel.S_APPLET_TOGGLE t, siebel.s_applet a, siebel.s_repository r
where t.APPLET_NAME like :applet||decode(:wc,'Y','%','') and vwti.APPLET_NAME = a.name and vwti.VIEW_WEB_TMPL_ID = vwt.row_id and vwt.VIEW_ID = v.row_id and v.name = sv.VIEW_NAME and sv.SCREEN_ID = s.row_id and t.applet_id = a.row_id
and a.repository_id = r.row_id and v.repository_id = r.row_id and s.repository_id = r.row_id and r.name = 'Siebel Repository'
and t.inactive_flg = 'N' and a.inactive_flg = 'N' and vwti.inactive_flg = 'N' and vwt.inactive_flg = 'N' and sv.inactive_flg = 'N' and v.inactive_flg = 'N'
union all
select 'Run Time Event - Object' attr_type, '' screen, '' viewname, rt.EVT_NAME obj_name, rt.OBJ_NAME attr_name, '' Applet_Mode from siebel.S_CT_EVENT rt
where rt.OBJ_NAME like decode(:wc,'Y','%','')||:applet||decode(:wc,'Y','%','') and rt.OBJ_TYPE_CD = 'Applet'

union all

select 'Personalization - Applet Condition' attr_type, '' screen, '' viewname, A.VIS_COND_EXPR obj_name, a.APPLET_NAME attr_name, '' Applet_Mode from siebel.S_CT_APPLET a
where a.APPLET_NAME like decode(:wc,'Y','%','')||:applet||decode(:wc,'Y','%','') and (a.EFF_END_DT is null or a.EFF_END_DT >= sysdate) and (a.EFF_START_DT is null or a.EFF_START_DT <= sysdate)
)

SQL Field Search

This is a Query to find references to a particular field.  Just replace the bind variables.  Use your imagination in modifying this query to find Profile attributes.  The objects searched for are those relevant to most Implementations and do not search more specialized features (such as tree applets for instance).  If those features are used, you will need to add another Union to the query.  Also, because of the limitations in searching Oracle LONG data types, this query does not search Script.  I keep a minimized repository search window open in my Tools session with all the script objects selected so that I can quickly drag the window up and paste in what I am searching for.

var :attr_val = "CMI Failure Id";             -- Field Name or string to search for

var :buscomp = "CMI Quote Simple";    -- Business Component of the relevant object if applicable
var :wc = "Y";                                            -- Trailing Wildcard.  N indicates an exact match

select attr_type, obj_name, attr_name, attr_val from (
select 'Applet List Column' attr_type, a.name obj_name, lc.name attr_name, lc.field_name attr_val from siebel.s_list_column lc, siebel.s_list l, siebel.s_applet a, siebel.s_repository r
where lc.field_name like :attr_val||decode(:wc,'Y','%','') and lc.list_id = l.row_id and l.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and lc.inactive_flg = 'N' and a.inactive_flg = 'N' and a.BUSCOMP_NAME = :buscomp

union all

select 'Applet Control' attr_type, a.name obj_name, c.name attr_name, c.field_name attr_val from siebel.s_control c, siebel.s_applet a, siebel.s_repository r
where c.field_name like :attr_val||decode(:wc,'Y','%','') and c.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and c.inactive_flg = 'N' and a.inactive_flg = 'N' and a.BUSCOMP_NAME = :buscomp

union all

select 'Applet User Prop' attr_type, a.name obj_name, up.name attr_name, up.Value attr_val from siebel.S_APPLET_UPROP up, siebel.s_applet a, siebel.s_repository r
where up.Value like '%'||:attr_val||'%' and up.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and up.inactive_flg = 'N' and a.inactive_flg = 'N' and a.BUSCOMP_NAME = :buscomp

union all

select 'Applet Toggle' attr_type, a.name obj_name, t.name attr_name, t.AUTO_TOG_FLD_NAME attr_val from siebel.S_APPLET_TOGGLE t, siebel.s_applet a, siebel.s_repository r
where t.AUTO_TOG_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and t.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and a.inactive_flg = 'N' and t.inactive_flg = 'N' and a.BUSCOMP_NAME = :buscomp

union all

select 'Applet Drilldown - Source' attr_type, a.name obj_name, d.name attr_name, d.SRC_FIELD_NAME attr_val from siebel.S_DDOWN_OBJECT d, siebel.s_applet a, siebel.s_repository r
where d.SRC_FIELD_NAME like :attr_val||decode(:wc,'Y','%','') and d.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository' and d.inactive_flg = 'N' and a.inactive_flg = 'N' and a.BUSCOMP_NAME = :buscomp

union all

select 'Applet Dynamic Drilldown' attr_type, a.name obj_name, d.name attr_name, dd.FIELD_NAME attr_val from siebel.S_DDOWN_DYNDEST dd, siebel.S_DDOWN_OBJECT d, siebel.s_applet a, siebel.s_repository r
where dd.FIELD_NAME like :attr_val||decode(:wc,'Y','%','') and dd.DDOWN_OBJECT_ID = d.row_id
and d.applet_id = a.row_id and a.repository_id = r.row_id and r.name = 'Siebel Repository'
and dd.inactive_flg = 'N' and d.inactive_flg = 'N' and a.inactive_flg = 'N' and a.BUSCOMP_NAME = :buscomp

union all

select 'Join Spec' attr_type, bc.name obj_name, j.name attr_name, js.SRC_FLD_NAME attr_val from siebel.S_JOIN_SPEC js, siebel.S_JOIN j, siebel.s_buscomp bc, siebel.s_repository r
where js.SRC_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and js.JOIN_ID = j.row_id and j.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and j.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'Predefault - Same' attr_type, bc.name obj_name, f.name attr_name, f.PREDEFVAL attr_val from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where f.PREDEFVAL like '%'||:attr_val||'%' and f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and f.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'Predefault - Other' attr_type, bc.name obj_name, f.name attr_name, f.PREDEFVAL attr_val from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where f.PREDEFVAL like '%'||:buscomp||'.'||:attr_val||'%' and f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and f.inactive_flg = 'N'

union all

select 'Calculated Field - Same' attr_type, bc.name obj_name, f.name attr_name, f.CALCVAL attr_val from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where f.CALCVAL like '%'||:attr_val||'%' and f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and f.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'Calculated Field - Other' attr_type, bc.name obj_name, f.name attr_name, f.CALCVAL attr_val from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where f.CALCVAL like '%ParentFieldValue%'||:attr_val||'%' and f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and f.inactive_flg = 'N'

union all

select 'BC User Prop' attr_type, bc.name obj_name, up.name attr_name, up.VALUE attr_val from siebel.S_BUSCOMP_UPROP up, siebel.s_buscomp bc, siebel.s_repository r
where up.VALUE like '%'||:attr_val||'%' and up.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and up.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'BC User Prop' attr_type, bc.name obj_name, up.name attr_name, up.VALUE attr_val from siebel.S_BUSCOMP_UPROP up, siebel.s_buscomp bc, siebel.s_repository r
where up.NAME like '%'||:attr_val||decode(:wc,'Y','%','') and up.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and up.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'BC User Prop - Child' attr_type, bc.name obj_name, up.name attr_name, up.VALUE attr_val from siebel.S_BUSCOMP_UPROP up, siebel.s_buscomp bc, siebel.s_repository r
where up.NAME like 'Parent%' and up.VALUE like '%'||:buscomp||'%'||:attr_val||'%' and up.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and up.inactive_flg = 'N' and bc.name <> :buscomp

union all

select 'BC User Prop - Child' attr_type, bc.name obj_name, up.name attr_name, up.VALUE attr_val from siebel.S_BUSCOMP_UPROP up, siebel.s_buscomp bc, siebel.s_repository r
where up.NAME like 'Parent%'||:buscomp and up.VALUE like :attr_val||decode(:wc,'Y','%','') and up.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and up.inactive_flg = 'N' and bc.name <> :buscomp

union all

select 'Pick Map - Same' attr_type, bc.name obj_name, f.name attr_name, pm.NAME attr_val from siebel.s_pickmap pm, siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r
where pm.FIELD_NAME like :attr_val||decode(:wc,'Y','%','') and pm.FIELD_ID = f.row_id and f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and f.inactive_flg = 'N' and pm.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'Pick Map - Other' attr_type, bc.name obj_name, f.name attr_name, pm.NAME attr_val from siebel.s_pickmap pm, siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r, siebel.S_PICKLIST pl
where pm.PICK_FIELD_NAME like :attr_val||decode(:wc,'Y','%','') and pm.FIELD_ID = f.row_id and f.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and r.row_id = pl.repository_id
and f.inactive_flg = 'N' and pm.inactive_flg = 'N' and f.PICKLIST_NAME = pl.name and pl.BUSCOMP_NAME = :buscomp

union all

select 'MVL Primary Id Field' attr_type, bc.name obj_name, mvl.name attr_name, mvl.PRIMEID_FLD_NAME attr_val from siebel.S_MVLINK mvl, siebel.s_buscomp bc, siebel.s_repository r
where mvl.PRIMEID_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and mvl.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and mvl.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'MVL Source Id Field' attr_type, bc.name obj_name, mvl.name attr_name, mvl.SRC_FLD_NAME attr_val from siebel.S_MVLINK mvl, siebel.s_buscomp bc, siebel.s_repository r
where mvl.SRC_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and mvl.BUSCOMP_ID = bc.row_id and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and mvl.inactive_flg = 'N' and bc.name = :buscomp

union all

select 'MVF Destination Field' attr_type, bc.name obj_name, f.name attr_name, mvl.DEST_BC_NAME||'.'||f.DEST_FLD_NAME attr_val from siebel.s_field f, siebel.s_buscomp bc, siebel.s_repository r, siebel.S_MVLINK mvl
where f.DEST_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and f.BUSCOMP_ID = bc.row_id and mvl.NAME = f.MVLINK_NAME and mvl.BUSCOMP_ID = bc.row_id and mvl.DEST_BC_NAME = :buscomp
and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and f.inactive_flg = 'N'

union all

select 'Link - Destination' attr_type, l.name obj_name, '' attr_name, l.DST_FLD_NAME attr_val from siebel.s_link l, siebel.s_repository r
where l.DST_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and l.CHILD_BC_NAME = :buscomp and l.repository_id = r.row_id and r.name = 'Siebel Repository' and l.inactive_flg = 'N'

union all

select 'Link - Search Spec' attr_type, l.name obj_name, '' attr_name, l.SRCHSPEC attr_val from siebel.s_link l, siebel.s_repository r
where l.SRCHSPEC like '%'||:attr_val||decode(:wc,'Y','%',']%') and l.CHILD_BC_NAME = :buscomp and l.repository_id = r.row_id and r.name = 'Siebel Repository' and l.inactive_flg = 'N'

union all

select 'Applet - Search Spec' attr_type, a.name obj_name, '' attr_name, a.SRCHSPEC attr_val from siebel.s_applet a, siebel.s_repository r
where a.SRCHSPEC like '%'||:attr_val||decode(:wc,'Y','%',']%') and a.BUSCOMP_NAME = :buscomp and a.repository_id = r.row_id and r.name = 'Siebel Repository' and a.inactive_flg = 'N'

union all

select 'Picklist - Search Spec' attr_type, p.name obj_name, '' attr_name, p.SRCHSPEC attr_val from siebel.S_PICKLIST p, siebel.s_repository r
where p.SRCHSPEC like '%'||:attr_val||decode(:wc,'Y','%',']%') and p.BUSCOMP_NAME = :buscomp and p.repository_id = r.row_id and r.name = 'Siebel Repository' and p.inactive_flg = 'N'

union all

select 'Bus Comp - Search Spec' attr_type, bc.name obj_name, '' attr_name, bc.SRCHSPEC attr_val from siebel.s_buscomp bc, siebel.s_repository r
where bc.SRCHSPEC like '%'||:attr_val||decode(:wc,'Y','%',']%') and bc.NAME = :buscomp and bc.repository_id = r.row_id and r.name = 'Siebel Repository' and bc.inactive_flg = 'N'

union all

select 'Action Set - Conditional Expression' attr_type, cas.name obj_name, ca.name attr_name, ca.COND_EXPR attr_val from siebel.S_CT_EVENT rt, siebel.s_ct_action ca, siebel.s_ct_action_set cas
where rt.CT_ACTN_SET_ID = cas.row_id and ca.CT_ACTN_SET_ID = cas.row_id and ca.COND_EXPR like '%'||:attr_val||decode(:wc,'Y','%',']%') and rt.OBJ_NAME = :buscomp and rt.OBJ_TYPE_CD = 'BusComp'

union all

select 'Action Set - Attribute Set' attr_type, cas.name obj_name, ca.name attr_name, ca.SET_RHS_EXPR attr_val from siebel.S_CT_EVENT rt, siebel.s_ct_action ca, siebel.s_ct_action_set cas
where rt.CT_ACTN_SET_ID = cas.row_id and ca.CT_ACTN_SET_ID = cas.row_id and ca.SET_RHS_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and rt.OBJ_NAME = :buscomp and rt.OBJ_TYPE_CD = 'BusComp'

union all

select 'Run Time Event - Conditional Expression' attr_type, rt.OBJ_NAME obj_name, rt.EVT_NAME attr_name, rt.ACTN_COND_EXPR attr_val from siebel.S_CT_EVENT rt
where rt.ACTN_COND_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and rt.OBJ_NAME = :buscomp and rt.OBJ_TYPE_CD = 'BusComp'

union all

select 'Run Time Event - Sub Event' attr_type, rt.OBJ_NAME obj_name, rt.EVT_NAME attr_name, rt.EVT_SUB_NAME attr_val from siebel.S_CT_EVENT rt
where rt.EVT_SUB_NAME like :attr_val||decode(:wc,'Y','%','') and rt.OBJ_NAME = :buscomp and rt.OBJ_TYPE_CD = 'BusComp'

union all

select 'Integration Component Field' attr_type, io.name obj_name, ic.name attr_name, icf.NAME attr_val from siebel.S_INT_FIELD icf, siebel.S_INT_COMP ic, siebel.S_INT_OBJ io, siebel.s_repository r
where icf.EXT_NAME like :attr_val||decode(:wc,'Y','%','') and icf.INT_COMP_ID = ic.row_id and ic.INT_OBJ_ID = io.row_id and io.repository_id = r.row_id and r.name = 'Siebel Repository'
and io.inactive_flg = 'N' and ic.inactive_flg = 'N' and icf.inactive_flg = 'N' and ic.EXT_NAME = :buscomp

union all

select 'IO Maps - Source Fields' attr_type, o.name obj_name, C.name attr_name, F.SRC_EXPR attr_val from siebel.S_INT_FLDMAP F, siebel.S_INT_COMPMAP C, siebel.S_INT_OBJMAP O, siebel.S_INT_OBJ IO,
siebel.S_INT_COMP IC, siebel.S_INT_FIELD ICF, siebel.s_repository r
where C.int_obj_map_id = O.row_id and F.int_comp_map_id = C.row_id and F.SRC_EXPR like '%'||ICF.NAME||'%' and IC.EXT_NAME like decode(:bcwc,'Y','%','')||:buscomp||decode(:bcwc,'Y','%','') and O.src_int_obj_name = IO.NAME and c.src_int_comp_name = IC.NAME
and ICF.ext_name = :attr_val||decode(:wc,'Y','%','') and IO.repository_id = r.row_id and r.name = 'Siebel Repository' and ICF.INT_COMP_ID = IC.row_id
and IC.INT_OBJ_ID = IO.row_id and io.inactive_flg = 'N' and ic.inactive_flg = 'N' and icf.inactive_flg = 'N' and (O.COMMENTS is null or O.COMMENTS not like 'ARCHIVE%')

union all

select 'IO Maps - Destination Fields' attr_type, o.name obj_name, C.name attr_name, F.dst_int_fld_name attr_val from siebel.S_INT_FLDMAP F, siebel.S_INT_COMPMAP C, siebel.S_INT_OBJMAP O, siebel.S_INT_OBJ IO,
siebel.S_INT_COMP IC, siebel.S_INT_FIELD ICF, siebel.s_repository r
where C.int_obj_map_id = O.row_id and F.int_comp_map_id = C.row_id and F.dst_int_fld_name = ICF.NAME and IC.EXT_NAME like decode(:bcwc,'Y','%','')||:buscomp||decode(:bcwc,'Y','%','') and O.dst_int_obj_name = IO.NAME and c.dst_int_comp_name = IC.NAME
and ICF.ext_name = :attr_val||decode(:wc,'Y','%','') and IO.repository_id = r.row_id and r.name = 'Siebel Repository' and ICF.INT_COMP_ID = IC.row_id
and IC.INT_OBJ_ID = IO.row_id and io.inactive_flg = 'N' and ic.inactive_flg = 'N' and icf.inactive_flg = 'N' and (O.COMMENTS is null or O.COMMENTS not like 'ARCHIVE%')

union all

select 'Data Maps - Source Fields' attr_type, o.name obj_name, C.name attr_name, f.src_fld_name attr_val from siebel.S_FIELD_DMAP F, siebel.S_BUSCOMP_DMAP C, siebel.S_BUSOBJ_DMAP O
where C.BUSOBJ_DMAP_ID = O.row_id and F.BUSCOMP_DMAP_ID = C.row_id and F.SRC_FLD_NAME like '%'||:attr_val||'%' and c.SRC_BUSCOMP_NAME = :buscomp

union all

select 'Data Maps - Destination Fields' attr_type, o.name obj_name, C.name attr_name, f.DST_FLD_NAME attr_val from siebel.S_FIELD_DMAP F, siebel.S_BUSCOMP_DMAP C, siebel.S_BUSOBJ_DMAP O
where C.BUSOBJ_DMAP_ID = O.row_id and F.BUSCOMP_DMAP_ID = C.row_id and F.DST_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and c.DST_BUSCOMP_NAME = :buscomp

union all

select 'WF Branch Criteria - Field' attr_type, wf.name obj_name, s.name attr_name, b.name attr_val from siebel.S_WFR_COND_CRIT cc, siebel.S_WFR_STP_BRNCH b, siebel.S_WFR_STP s, siebel.S_WFR_PROC wf, siebel.s_repository r
where cc.BUSCOMP_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and cc.BRANCH_ID = b.row_id and b.STEP_ID = s.row_id and s.PROCESS_ID = wf.row_id and wf.repository_id = r.row_id and r.name = 'Siebel Repository'
and wf.STATUS_CD = 'COMPLETED' and cc.BUSCOMP_NAME = :buscomp

union all

select 'WF Branch Criteria - Expression' attr_type, wf.name obj_name, s.name attr_name, b.name attr_val from siebel.S_WFR_COND_VAL cv, siebel.S_WFR_COND_CRIT cc, siebel.S_WFR_STP_BRNCH b, siebel.S_WFR_STP s, siebel.S_WFR_PROC wf, siebel.s_repository r
where cv.LO_CHAR5 like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and cv.COND_CRIT_ID = cc.row_id and cc.BRANCH_ID = b.row_id and b.STEP_ID = s.row_id and s.PROCESS_ID = wf.row_id and wf.repository_id = r.row_id and r.name = 'Siebel Repository'
and wf.STATUS_CD = 'COMPLETED' and cc.BUSCOMP_NAME = :buscomp

union all

select 'WF Branch Step - Argument' attr_type, wf.name obj_name, s.name attr_name, a.name attr_val from siebel.S_WFR_STP_ARG a, siebel.S_WFR_STP s, siebel.S_WFR_PROC wf, siebel.s_repository r
where a.BUSCOMP_FLD_NAME like :attr_val||decode(:wc,'Y','%','') and a.STEP_ID = s.row_id and s.PROCESS_ID = wf.row_id and wf.repository_id = r.row_id and r.name = 'Siebel Repository'
and wf.STATUS_CD = 'COMPLETED' and a.BUSCOMP_NAME = :buscomp

union all

select 'DVM - Rule' attr_type, rs.name obj_name, r.name attr_name, r.RULE_EXPR attr_val from siebel.S_VALDN_RULE R, siebel.S_VALDN_RL_SET RS
where R.RULE_SET_ID = rs.row_id and r.RULE_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and r.BUSCOMP_NAME = :buscomp and rs.status_cd = 'Active'

union all

select 'Personalization - Applet Condition' attr_type, a.APPLET_NAME obj_name, a.APPLET_NAME attr_name, A.VIS_COND_EXPR attr_val from siebel.S_CT_APPLET a, siebel.S_APPLET ra, siebel.s_repository r
where a.APPLET_NAME = ra.name and a.VIS_COND_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and r.name = 'Siebel Repository' and ra.repository_id = r.row_id and ra.inactive_flg = 'N'
and ra.buscomp_name like decode(:bcwc,'Y','%','')||:buscomp||decode(:bcwc,'Y','%','') and ra.INACTIVE_FLG = 'N' and (a.EFF_END_DT is null or a.EFF_END_DT >= sysdate) and (a.EFF_START_DT is null or a.EFF_START_DT <= sysdate)

union all

select 'Personalization - Applet Rule Set' attr_type, a.APPLET_NAME obj_name, to_char(ar.SEQ_NUM) attr_name, ar.COND_EXPR attr_val from siebel.S_CT_APPLET a, siebel.S_CT_APLT_RLST ar, siebel.S_APPLET ra, siebel.s_repository r
where a.APPLET_NAME = ra.name and ar.ct_applet_id = a.row_id and ar.COND_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and r.name = 'Siebel Repository' and ra.inactive_flg = 'N' and ra.repository_id = r.row_id
and ra.buscomp_name like decode(:bcwc,'Y','%','')||:buscomp||decode(:bcwc,'Y','%','') and ra.INACTIVE_FLG = 'N' and (a.EFF_END_DT is null or a.EFF_END_DT >= sysdate) and (a.EFF_START_DT is null or a.EFF_START_DT <= sysdate)
and (ar.EFF_END_DT is null or ar.EFF_END_DT >= sysdate) and (ar.EFF_START_DT is null or ar.EFF_START_DT <= sysdate)

union all

select 'Personalization - Rule Condition' attr_type, rs.name obj_name, r.Name attr_name, r.COND_EXPR attr_val from siebel.S_CT_RULE R, siebel.S_CT_RULE_SET RS, siebel.S_CT_APPLET a, siebel.S_CT_APLT_RLST ar, siebel.S_APPLET ra, siebel.s_repository rep
where a.APPLET_NAME = ra.name and ar.ct_applet_id = a.row_id and rep.name = 'Siebel Repository' and ra.repository_id = rep.row_id and ra.inactive_flg = 'N' and rs.row_id = ar.ct_rule_set_id
and R.CT_RULE_SET_ID = rs.row_id and r.COND_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%')
and ra.BUSCOMP_NAME = :buscomp and rs.ACTIVE_FLG = 'Y' and r.ACTIVE_FLG = 'Y' and ra.INACTIVE_FLG = 'N' and (a.EFF_END_DT is null or a.EFF_END_DT >= sysdate) and (a.EFF_START_DT is null or a.EFF_START_DT <= sysdate)
and (ar.EFF_END_DT is null or ar.EFF_END_DT >= sysdate) and (ar.EFF_START_DT is null or ar.EFF_START_DT <= sysdate)

union all

select 'Personalization - Rule Include Condition' attr_type, rs.name obj_name, r.Name attr_name, r.INCL_RULE_EXPR attr_val from siebel.S_CT_RULE R, siebel.S_CT_RULE_SET RS, siebel.S_CT_APPLET a, siebel.S_CT_APLT_RLST ar, siebel.S_APPLET ra, siebel.s_repository rep
where a.APPLET_NAME = ra.name and ar.ct_applet_id = a.row_id and rep.name = 'Siebel Repository' and ra.repository_id = rep.row_id and ra.inactive_flg = 'N' and rs.row_id = ar.ct_rule_set_id
and R.CT_RULE_SET_ID = rs.row_id and r.INCL_RULE_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%')
and ra.BUSCOMP_NAME = :buscomp and rs.ACTIVE_FLG = 'Y' and r.ACTIVE_FLG = 'Y' and ra.INACTIVE_FLG = 'N' and (a.EFF_END_DT is null or a.EFF_END_DT >= sysdate) and (a.EFF_START_DT is null or a.EFF_START_DT <= sysdate)
and (ar.EFF_END_DT is null or ar.EFF_END_DT >= sysdate) and (ar.EFF_START_DT is null or ar.EFF_START_DT <= sysdate)

union all

select 'Personalization - Rule Exclude Condition' attr_type, rs.name obj_name, r.Name attr_name, r.EXCL_RULE_EXPR attr_val from siebel.S_CT_RULE R, siebel.S_CT_RULE_SET RS, siebel.S_CT_APPLET a, siebel.S_CT_APLT_RLST ar, siebel.S_APPLET ra, siebel.s_repository rep
where a.APPLET_NAME = ra.name and ar.ct_applet_id = a.row_id and rep.name = 'Siebel Repository' and ra.repository_id = rep.row_id and ra.inactive_flg = 'N' and rs.row_id = ar.ct_rule_set_id
and R.CT_RULE_SET_ID = rs.row_id and r.EXCL_RULE_EXPR like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%')
and ra.BUSCOMP_NAME = :buscomp and rs.ACTIVE_FLG = 'Y' and r.ACTIVE_FLG = 'Y' and ra.INACTIVE_FLG = 'N' and (a.EFF_END_DT is null or a.EFF_END_DT >= sysdate) and (a.EFF_START_DT is null or a.EFF_START_DT <= sysdate)
and (ar.EFF_END_DT is null or ar.EFF_END_DT >= sysdate) and (ar.EFF_START_DT is null or ar.EFF_START_DT <= sysdate)

union all

select 'Batch Job - Search' attr_type, a.DISPLAY_NAME obj_name, wf.NAME attr_name, ps.VALUE attr_val
from siebel.S_SRM_REQUEST q, siebel.S_SRM_ACTION a, siebel.S_SRM_REQ_PARAM ps, siebel.S_SRM_ACT_PARAM psp, siebel.S_SRM_REQ_PARAM pw, siebel.S_SRM_ACT_PARAM pwp, siebel.S_WFR_PROC wf, siebel.s_repository r, siebel.S_BUSOBJ bo
where ps.REQ_ID = q.row_id and ps.VALUE like decode(:wc,'Y','%','%[')||:attr_val||decode(:wc,'Y','%',']%') and q.STATUS = 'ACTIVE' AND q.REQ_TYPE_CD = 'RPT_PARENT' and ps.ACTPARAM_ID = psp.ROW_ID
and psp.NAME = 'Search Specification' and pw.REQ_ID = q.row_id and pw.ACTPARAM_ID = pwp.ROW_ID and pwp.NAME = 'Workflow Process Name' and a.row_id = q.action_id and pw.value = wf.PROC_NAME and wf.REPOSITORY_ID = r.row_id
and r.name = 'Siebel Repository' and bo.repository_id = r.row_id and wf.STATUS_CD = 'COMPLETED' and wf.BUSOBJ_NAME = bo.name and bo.PR_BUSCOMP_NAME like decode(:bcwc,'Y','%','')||:buscomp||decode(:bcwc,'Y','%','')
);

Tuesday, January 3, 2012

My BI Quick Reference

I know there are a bunch of cheat sheets out there, but I frequently don't find everything I am looking for in one place so figured I would just start building my own.

Siebel Functions

Include statement

<?namespace:psfn=http://www.oracle.com/XSL/Transform/java/com.siebel.xmlpublisher.reports.XSLFunctions?>

Date Conversion

<?psfn:totext(OrderDate,"MM/dd/yyyy","MM/dd/yyyy hh:mm:ss")?>

Loops:

Basic Loop where QuoteItem is the Integration Component/XML Group
<?for-each:QuoteItem?>
Add a where clause to constrain rows in the loop.  Multiple constraints can be added back to back with each bracketed section representing an AND.  An OR would need to be done inside a single bracketed expression.  the .// is an xpath expression to determine the XML group of the field
<?for-each:QuoteItem[.//LineType='Sales']?>
<?for-each:QuoteItem[.//LineType='Sales'][.//LineNumber<100]?>
<?for-each:QuoteItem[.//LineType='Sales' or .//LineType='Service']?>
Groupings 

Loop that groups by the column LineType and sorts by the grouping
<?for-each-group:QuoteItem;./LineType?><?sort:current-group()/LineType;'ascending';data-type='text'?>
A nested loop showing the sub group records for the loop above:
<?for-each:current-group()?><?sort:Product;'ascending';data-type='text'?>

Loop that groups records together by a particular column and makes each grouping a column in a table
<?for-each-group@column:QuoteItem;./LineType?>
Similar to grouping by column is to group by section.  In this case each grouping creates a heading when the report breaks across multiple pages.
<?for-each@section:G_CUSTOMER?>

Group Expressions

This expression is used to sum a column from series of records outside the context of a loop.  The expression in the bracket specifies those records to include, in this case only non null values.  This is an XPath expression.
<?sum(.//ItemExtendedPriceTotal[.!=''])?>
Conditionals:

If/else
<?if:Comment!=''?>
<?Comment?>
<?end if?>

Keep in mind that Carriage returns outside the expression will still appear so consider this when judging where to put the donditional

Switch/Case/Select/Choose:

<?choose:?> <?when:MY_FIELD='value tested'?> <?call:template?> <?end when?> <?when:MY_FIELD_2='value tested 2'?> <?call:template_2?> <?end when?> <?otherwise:?> <?call:template_other?> <?end otherwise?> <?end choose?> 

Embedding a 64 Bit Image

For this to work, there must be a 64 Bit attachment embedded in the Integration Object being sent to BI. In this example,QuoteAttachment is the element name of the IC Field of type DTYPE_ATTACHMENT containing the 64 Bit inline image.
<fo:instream-foreign-object content-type="image/jpg"><?QuoteAttachment?></fo:instream-foreign-object>
Additional attributes for the fo:instream-foreign-object tag in addition to content-type can resize the image: Specify in pixels as follows:

<fo:instream-foreign-object content type="image/jpg" height="300 px" width="4 px">

... or in centimeters:
<fo:instream-foreign-object content type="image/jpg" height="3 cm" width="4 cm">
... or as a percentage of the original dimensions:
<fo:instream-foreign-object content type="image/jpg" height="300%" width="300%"> ...