Oracle

Using sed to clean up an LDIF file for import #Oracle #Identity #UNIX

I needed to import a group of users, into Oracle Internet Directory (OID) with attributes in a variety of backend data stores. I used Oracle Virtual Directory to virtualize the data stores into a single ldap view. I used the OVD adapter configuration to specify which attributes I wanted returned. I then exported using the export control from Apache Directory Studio. This resulted in an ldif file containing all of the records I needed with attributes. There were a few additional attributes as a result of using OVD that I now had to deal with.

I ended up with an ldif file that contained a lot of records like this:

dn: cn=Babs Jensen@ACME.GOV,ou=temp_user_load
objectclass: inetOrgPerson
objectclass: organizationalPerson
objectclass: person
objectclass: top
cn: 1234556677@ACME.GOV
cn: Babs.Jensen@ACME.gov
cn: Jensen, Babs
sn: Jensen
givenName: Babs
mail: BABS.JENSEN@ACME.GOV
vdejoindn: ou=acmeinfo_temp:cn=JENSEN\,BABS,ou=acmeinfo_temp
vdejoindn: AD_temp:CN=babs.jensen@ACME.GOV,OU=locations,OU=park,ou=ad_t
emp,dc=acme,dc=local
fascnDecoded: 1234567890987654321
guid: ABcdedghi1234567890
ssn: 12345678

…

Note: With the SED command you can make changes directly to the source file but I am creating a new target file with each change I can make so that I can always revert back if the command doesn’t work exactly the way I want it to.

I wanted to get rid of lines that don’t start with an attribute name (In my case I am free to get rid of lines that carry over into the second line … YMMV)

I also wanted to specifically wanted to get rid of all lines that start with “vdejoindn:” and there are also some vdejoindn lines that overrun onto a second line that won’t beremoved if I use sed to remove lines with the pattern matching vdejoindn:.

So, first I want to remove all lines that don’t contain a colon. This removes the overrun lines but also all blank lines.

$ sed ‘/:/!d’ input.ldif > tmp.ldif

this keeps the lines with a colon.

But now we don’t have breaks between the records

$ sed ‘s/^dn:/\n&/g’ tmp.ldif > tmp2.ldif

Ok, now I want to get rid of the lines that have “vdejoindn:”.

$ sed ‘/vdejoindn:/d’ tmp2.ldif > tmp3.ldif

Now at some point I ended up with “^M” at the end of each file … I don’t know if this is because I opened with VIM in Windows before moving to Linux … I am going to assume so but either way in this instance I want to remove these characters.

$ dos2unix tmp3.ldif > tmp4.ldif

Alright, Now, for me to import this into Oracle Internet Directory (OID) I’ll need to add the “changetype” directive. I am going to add the string “changetype: add” on a new line after each line with “ou=temp_user_load:” which is the temporary suffix I used in this export.

$ sed ‘/ou=temp_user_load/ a\changetype: add’ tmp4.ldif > tmp5.ldif

Now, should be the last step, prior to importing, is to correct the entries “DN” attribute. Essentially, we need to replace “ou=temp_user_load” with the correct suffix for where these users will be created.

$ sed ‘s/ou=temp_user_load/cn=Users,o=icam,dc=acme,dc=local/g’ tmp5.ldif > tmp6.ldif

At this point my ldif file (“tmp6.ldif”) is ready to import into my directory. You can use the ldapmodify command or since I am using OID you can use bulkload (which is recommended for large record sets).

Troubleshooting errors starting #OID #11g #Oracle #Identity #LDAP

I have an Oracle Identity 11g environment running on VirtualBox 4.0. This is a development environment that I use to test out various installations and configurations. I noticed the other day that I wasn’t able to start the Oracle Internet Directory (OID) instance.

When I checked the log file I can see that I am not able to connect to the Database. By the way, the log that is referenced doesn’t show anything of value. The log that actually contained the error is called: oidmon-0000.log

According to ora-code.com ora-28000 the error means that the user account that is connecting to the database ‘ODS’ is locked.

ORA-28000:	the account is locked
Cause:	The user has entered wrong password consequently for maximum number of times specified by the user’s profile parameter FAILED_LOGIN_ATTEMPTS, or the DBA has locked the account
Action:	Wait for PASSWORD_LOCK_TIME or contact DBA

It’s typically trivial to unlock an account from the sqlplus command line

So, we should be good now. I will try to start the process again.

But now my log shows

So, now I am getting an ORA-01017 error. Which means “Invalid username/password”. So, it seems that the Database doesn’t like the password that OID is supplying to connect to the ODS schema.

I’ll use SQL Developer to try and connect to the database with the ODS user

Interesting, SQL Developer is showing an ORA-28000 error.

Let’s try connecting using SQLPlus …

So, it seems we have a consensus (and yes, I did just include my password in the screenshot … it doesn’t matter)

Let’s see what the database has to say about this user. Make sure you reconnect to the DB as oracle.

Ok, didn’t we just unlock it? Let’s try again …

So, now what is the status?

Hey! This is good right? … the account seems to be open again.

So, let’s try to start OID again.

Ok, this is looking pretty ugly right about now…

… and the account is locked again. So, let’s see if we can figure out why this is happening.

Maybe the wallet that holds the ODS password for OID has become corrupt. We can recreate it using oidpasswd.

Note: Before you run oidpasswd it’s important to have your Oracle environment set up correctly. Here is what I am using (yours may vary):

ORACLE_SID=orcl

ORACLE_BASE=/opt/oracle

ORACLE_INSTANCE=/opt/oracle/Middleware/asisnt_1

ORACLE_HOME=/opt/oracle/Middleware/Oracle_IDM1

MW_HOME=/opt/oracle/Middleware

Now with this output … I have verified the location of the tnsnames.ora file and the information in it … so I am going to assume for the moment that the issue is with the password (at least until I prove otherwise).

Typically, changing the password will unlock the account

But here we are and the account is still locked.

… I am spending some time just fishing around on the Internet and looking around at my system

Wait a second … I wasn’t even thinking about ODSSM …

Change the ODSSM’s password and then unlock ODS.

So, both accounts should now be “OPEN”

Now restart the OIDMON process

What does the log say

Completely different error this time. At least I feel like we are making some progress …

hmmm … if the wallet can’t be read … maybe we can recreate the wallet. Let’s re-run the “create wallet” command that we tried earlier.

Hey! … it was successful this time. So, let’s try starting the OID processes

That was successful!

Now to check the status of the OPMN Processes

All of the OID related processes are now Alive. The ohs1 process is down because I turned it off earlier.

Recover Weblogic server admin password (on Linux)

On a virtual machine that I installed Oracle Identity Federation I found that I could not remember what I  had set the Weblogic Server (WLS) password to.  I needed a way to recover this password so that I would not have to reinstall WLS.  This isn’t the first time I have forgotten the password to start and login to WLS … I needed to find a reusable solution that would give me the password quickly. I found Kenneth Xu’s blog (“Program It”) where he defined a solution, in great detail.  Kenneth’s solution was geared towards Windows … I needed a solution for Linux (fortunately there were very minor changes required).   In other words … I borrowed heavily from: http://kennethxu.blogspot.com/2006/04/how-to-recover-weblogic-admin-password.html

Step 1:  On the Linux server (I am logged in as Oracle) create a development directory

I created one called: /home/oracle/deve

Step 2:  Create a file called:  RecoverPassword.java and then copy in the following code

import weblogic.security.internal.BootProperties;
public class RecoverPassword {
public static void main(String[] args) {
String BPF =          
"/opt2/oracle/Middleware/user_projects/domains/IDMDomain/servers/wls_oif1/data/
nodemanager/boot.properties";
BootProperties.load(BPF, false);
BootProperties bootp = BootProperties.getBootProperties();
System.out.println(                             
"##############################[" + bootp.getOneClient() + 
         "/" + bootp.getTwoClient() + "]#############################");   } }

Step 3:  Compile:

javac -classpath /opt2/oracle/Middleware/wlserver_10.3/server/lib/weblogic.jar RecoverPassword.java

Step 4:  Copy WLS Startup File to development directory

cp /opt2/oracle/Middleware/user_projects/domains/IDMDomain/bin/startWebLogic.sh .

Step 5: Rename to: recoverPassword.sh

Step 6:  Edit recoverPassword.sh

${JAVA_HOME}/bin/java ${JAVA_VM} -version // this is an existing line
### Custom Code inserted to Recover Password ###
CLASSPATH=/home/oracle/deve/:$CLASSPATH; export CLASSPATH
echo $CLASSPATH
SERVER_CLASS=RecoverPassword; export SERVER_CLASS
doExitFlag=false; export doExitFlag
if [ "${WLS_REDIRECT_LOG}" = "" ] ; then // this is en existing line

Step 7:  Change to the domain home directory

cd /opt2/oracle/Middleware/user_projects/domains/IDMDomain/

Step 8:  Run the recoverPassword.sh script

/home/oracle/deve/recoverPassword.sh

Output will look like:

ware/Oracle_IDM1 -Xms512m -Xmx1024m -Xss512K -Djava.net.preferIPv6Addresses=true -DuseIPv6Address=true -Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dweblogic.management.discover=false -Djava.net.preferIPv6Addresses=true -Dweblogic.management.discover=true  -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext.dirs=/opt2/oracle/Middleware/patch_wls1032/profiles/default/sysext_manifest_classpath

RecoverPassword

##############################[weblogic/Passw0rd1]#############################

The password is displayed on the line with the hashmarks.

Upgrade #Oracle #OIF to 11.1.1.3 #IDM #Identity

We installed Oracle Identity Federation (OIF) 11.1.1.2 a few months ago and had to move on to some other, more pressing IDM-related issues.  We finally came back to the Federation tasks at the beginning of September.  The first thing I did was take an inventory of where we left off and compared to what the current released version was from Oracle.  I found that we were now a version behind with both Weblogic Server (WLS) and OIF.  I initially put off upgrading because we were in a hurry to integrate with one of their business partners.  We were able to configure the Circle of Trust with the Relying Party (RP, aka Service Provider) with just a few issues.   This particular partner is using OpenSAML as their software of choice.  The only issue for us is that they didn’t (or don’t) create metadata files.  This is their choice because OpenSAML has a module for doing this.  The metadata files is a feature in SAML 2.0 that allows for easy (…easier) integration with your Federation partners.  I was able to create one manually for them by using the sp.xml file that was created when using the OpenSSO Fedlet (that’s for another post).

So, finally on to the point of this post.  The only issues that we have had with OIF 11.1.1.2 is that when trying to search for local users (we are using OVD as our User Data Store … OVD front’s two different AD instances) we have some issues with the search function and not all users can authenticate.  Yes, this is actually a major problem.

I noticed via http://support.oracle.com that there are a lot of patches available for 11.1.1.2.  I ended up downloading the 11.1.1.3 version from OTN (here).

(Note:  I talked to my contact at Oracle Support who said that 11.1.1.4 is coming very soon)

This version requires that Weblogic be at least 10.1.3.  I went back to the support site and downloaded the 10.1.3 patch from there.  It’s a jar file that is run and will open up as an OUI installer.  I found this site which I used as a guide.  It’s pretty simple and painless.  Make sure that you restart WLS after upgrading and before upgrading OIF.  When the OIF upgrade is complete you should restart the managed service.

After restarting OIF I noticed in Enterprise Manager (EM) that OIF is still displaying as 11.1.1.2.  I am running the Upgrade Assistant (Oracle_Home/bin/ua).  On the second screen you can select “Verify Instance”.  This will walk you through and verify that your OIF instance is upgraded to the correct version.  In my case the status is showing as “Failed”.    One thing that seems odd to me is that the port shown (on the error message) is 7499.  It looks like it’s trying to access the URL to the metadata file and is trying to go on 7499. (i.e., http://hostname:7499/fed/idp/metadata).  I can get to the file via 7777 and not 7499.  So, I’ll need to check later as to why the Upgrade Assistant is using that port.

I just tried to re-run the 11.1.1.3 patch installer.  It complained that the patch had already been applied to this Oracle_Home.  So, now I am perplexed.  Let’s try rebooting the box and restarting the WLS and OIF services.

Interestingly, after the reboot the OIF version is still showing as 11.1.1.2 … but my OIF LDAP Authentication Engine error is no longer occurring.  So, maybe it did get patched??  I am working on confirming this … maybe the version number doesn’t get updated?  … that doesn’t sound right though.

Directory Integration Platform (DIP w/Oracle DB #oracle #idm #dip #oid

DBImport Profile: (sync w/Oracle DB table)

Important Notes:

• Diptester utility has not been updated for 11g yet.
• Oracle recommends setting up sync profile to DB by using command line tools instead of EM UI.

Where is the profile stored:

Profile DN: orclodipagentname=”profile name”,cn=subscriber profile,cn=changelog subscriber,cn=oracle internet directory

Which files make up the profile:

Profile Name: “Profile Name”
Properties File: “profile name”.properties (Connection Info)
Configuration: “profile name”.cfg.master (SQL Query)
Map File: “profile name”.map.master (map columns to attributes)

DIP Log Location:
[/apps/oracle/]Middleware/Oracle_IDM1/ldap/odi/conf
Updating DBImport Profile: (After making changes to the config files you will need to update the profile)

$ manageSyncProfiles update -h hostname-p 7005 -D weblogic -pF “profile name”-f “profile properties file name”

Warning : Setting an incorrect value for the last change number could cause the profile to stop working or cause undesired sync operations
Do you want to continue [Y/N] Y
[Connected Directory Password] // Source password … DB user password in this case.
[Weblogic user password]
Connection parameters initialized.
Connecting at hostname:7005, with userid “weblogic”..
Connected successfully.

Profile “profile name” successfully updated.

Activate / Deactivate Profiles:

manageSyncProfiles activate -h dip_hostname -p 7005 -D weblogic -pf “profile name”
manageSyncProfiles deactivate -h dip_hostname-p 7005 -D weblogic -pf “profile name”
View the Subscriber Profile:
ldapsearch -h dip_hostname -p 3060 -D “cn=orcladmin” -w “orcladmin_password” -b “orclodipagentname=”profile name”,cn=subscriber profile,cn=changelog subscriber,cn=oracle internet directory” -s sub ‘objectclass=*’

Update the orclodipcondirlastappliedchgnum attribute in the sync profile:

// if there are any problems with the initial sync and you have to do it over … you will need to reset the last applied change number in the profile.

ldapmodify -h [diphostname] -p 3060 -D “cn=orcladmin” -w Passw0rd1 -f update_timestamp.ldif

update_timestamp.ldif:
dn: orclodipagentname=[diphostname],cn=subscriber profile,cn=changelog subscriber,cn=oracle internet directory
changetype: modify
replace: orclodipcondirlastappliedchgnum
orclodipcondirlastappliedchgnum: 20000101120000

Installing and Configuring Sun #DSEE 6.3 #Oracle #IDM #DS

To quickly install and configure Sun’s Oracle’s Directory Server Enterprise Edition (DSEE) 6.3:

1. Unpack the download (tar -xvzf)
2. CD into the DSEE_ZIP_DISTRIBUTION folder
3. Install the software:./dsee_deploy install -i /opt/ds6
4. Create a new instance:./dsadm create -p port -P SSL-port instance-path
5. Start the new instance:./dsadm start instance-path
6. Modify the example.ldif and then import (to create a new suffix)./dsconf create-suffix -h localhost -p 1389 dc=example,dc=com

   ./dsconf import -h localhost -p 1389 install-path/ds6/ldif/Example.ldif \ dc=example,dc=com

Debugging OVD 11g Installation #Oracle #IdM #OVD

I am installing Oracle Virtual Directory 11g on Oracle Enterprise Linux 5.  As the OVD installer ran through it’s Configuration Tools it stumbled on the Create ASInstance step with the following error:

Error creating ASInstance asinst_1.
Cause:
An internal operation has failed.  Unable to validate
NonJ2EEManagement Application deployment on admin server.
See logs for more details.

I thought maybe I had forgotten to start Weblogic, but that is not the case.

I will be working through this issue this afternoon … possibly this evening depending on how this afternoon goes.  If you have seen this before drop a commant, I’d love to here some feedback.

<Update>

Deborah Volk pointed out that I needed to use the latest version of Weblogic server.  I believe that is cased closed.

Oracle OAM Identity Server errors starting on OEL5 #Oracle #IDM #OAM

Hardware Load Balancers #Oracle #IdM #OAM

Many people ask how they should implement hardware-based load balancers into an Oracle Access Manager (OAM) implementation. As a rule of thumb: load balancers should be placed in front of the web/application servers where the web gates are installed (i.e., between the browser and the web server) and that is it. OAM has built-in load balance functionality and can manage fail-over on its own. This is referred to as software-based load balancing.

I have seen some environments where clients have put load balancers in between the Identity/Access servers and their directory servers. With OAM this is unnecessary and not advised. According to Oracle documentation,

“performance can be negatively affected by the load balancer, which can terminate a connection but fail to trigger a response that OAM can adjust to. This can cause outages.”

- Performance Tuning 3-39,  http://download.oracle.com/docs/cd/E12530_01/oam.1014/e10353.pdf

Troubleshooting a Forms Based AuthN Scheme #Oracle #IdM #OAM

The best part about my job is that I am constantly in a position to learn new things.  I like to learn and discover new things.  While troubleshooting a form-based authentication scheme in Oracle Access Manager (OAM) I found that you can test the login (i.e., verify the credentials) and bypass the  login form by entering the the protected url with the login and password in the query string  in the browser address bar.  This is a really handy way to make sure that the credentials you are using are actually valid (and as I learned as a kid watching cartoons after school … knowing is half the battle).

i.e.,

http://server/protected/page.html?login=jsmith&password=MyPwd

Here is the snippet from Oracle’s documentation:

To make sure that the authentication scheme is set properly, you can attempt to access a resource protected with that authentication scheme, adding the credentials as query string parameters. This simulates a form whose method is GET without actually using the form.

For example, suppose the authentication scheme uses the following creds challenge parameter:

creds:login password

In this example, if the protected URL is http://server/protected/page.html, you could launch a browser instance and type the following:

http://server/protected/page.html?login=jsmith&password=MyPwd