From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump

Description:

from an LDAP directory dump. Michael Doran, Systems Librarian ... dumps records. The 'callback' option allows you to process. each record as it is received ... – PowerPoint PPT presentation

Number of Views:249
Avg rating:3.0/5.0
Slides: 54
Provided by: rock9
Learn more at: http://rocky.uta.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump


1
From LDAP to Patron Load Generating a patron
update SIF file from an LDAP directory dump
  • Michael Doran, Systems Librarian

Endeavor Users Group Meeting, Chicago,
IL Session 8, Thursday, April 26, 2007
2
Integrated library system
patron database
Michael Doran 123 Elm St. Arlington, TX PATRON
BLOCK
patron DB
3
Getting patron data into Voyager
Doh!
  • Manual data entry via the Circulation client

4
Getting patron data into Voyager
  • Manual data entry via the Circulation client
  • Batch loading via the Pptrnupdt job
  • Technical Users Guide
  • 6 Patron Update
  • 19 Patron Record
  • Standard Interface File

5
The Patron SIF specification
  • Each record will consist of
  • 1 base segment - fixed length
  • 52 fixed length fields
  • certain fields are required
  • 1-9 address segments - fixed length
  • 19 fixed length fields
  • one address is required
  • 1 optional notes segment - variable length
  • 1 variable length field
  • 1 end-of-record mark

6
Patron SIF one record
7
Patron SIF one record
8
Patron SIF documentation
9
Patron SIF documentation
Item (field) number 72 total
Item (field) name
Offset starting point
Item (field) description
Format n - numeric s - string d - date
Required field?
Length of field
10
Patron SIF files the easy way
You send the Patron SIF documentation to your
University IT department
Heres your Patron SIF file. Enjoy!
Patron SIF
programmer
and voila
11
Patron SIF files the hard way
I want you to write a program that generates
Patron SIF files. Enjoy!
Voyager Manuals
your boss
12
So we have our marching orders
13
Somewhere on your campus is
  • One or more directories of information on people
    registered to attend classes
  • One or more directories of information on people
    who work for the university

students
faculty and staff
patrons
14
Directories, databases, protocols, APIs
  • directory
  • an alphabetical or classified list (as of names
    or addresses)
  • database
  • a collection of data organized for rapid
    retrieval (as by a computer)
  • protocol
  • a standard that controls the connection,
    communication, and data transfer between two
    computing endpoints
  • application programming interface (API)
  • an abstraction layer between the gory details of
    an application and the world outside that
    software

15
Directories, databases, protocols, APIs
LDAP directory
  • directory
  • an alphabetical or classified list (as of names
    or addresses)
  • database
  • a collection of data organized for rapid
    retrieval (as by a computer)
  • protocol
  • a standard that controls the connection,
    communication, and data transfer between two
    computing endpoints
  • application programming interface (API)
  • an abstraction layer between the gory details of
    an application and the world outside that
    software

Lightweight Directory Access Protocol
16
Directories, databases, protocols, APIs
Voyager patron load
  • directory
  • an alphabetical or classified list (as of names
    or addresses)
  • database
  • a collection of data organized for rapid
    retrieval (as by a computer)
  • protocol
  • a standard that controls the connection,
    communication, and data transfer between two
    computing endpoints
  • application programming interface (API)
  • an abstraction layer between the gory details of
    an application and the world outside that
    software

Standard Interface File (SIF)
Pptrnupdt
17
Voyager patron database tables and relationships
18
The pieces and players
Voyager
Directory
LDAP Protocol
LDAP client
NetLDAP
LDAP server
Perl program
?
patron SIF
Pptrnupdt
patron info
patron info
19
The pieces and players
Voyager
Directory
LDAP Protocol
LDAP client
NetLDAP
LDAP server
Perl program
?
patron SIF
Pptrnupdt
patron info
patron info
patron info
20
Three easy(?) steps
  • Query the LDAP directory
  • filter for valid patrons
  • Process the records
  • minor data formatting
  • decision algorithms
  • output records in SIF format
  • Load SIF file into Voyager
  • Pptrnupdt

21
Three easy(?) steps
  • Query the LDAP directory
  • filter for valid patrons
  • Process the records
  • minor data formatting
  • decision algorithms
  • output records in SIF format
  • Load SIF file into Voyager
  • Pptrnupdt

Perl program
22
Before we start Perl programming
Voyager
Directory
LDAP Protocol
LDAP client
NetLDAP
LDAP server
ldapsearch
Perl program
?
patron SIF
Pptrnupdt
patron info
patron info
23
What you need to get started
  • command line client
  • ldapsearch comes standard on Solaris
  • LDAP server info
  • hostname
  • port numbers
  • 389 default ldap port
  • 636 default ldaps port
  • LDAP Schema
  • bind credentials
  • Distinguished Name (DN)
  • anonymous
  • your own (e.g. NetID username password)
  • superuser (provided by LDAP administrator)

24
ldapsearch search
ldapsearch \ -h "ldap.uta.edu" \ -D
"uiddoran,cnaccounts,dcuta,dcedu" \ -w
"ltpasswordgt" \ -b "cnpeople, dcuta, dcedu" \
"utaAccountNamedoran"
25
ldapsearch search
ldapsearch \ -h "ldap.uta.edu" \ -D
"uiddoran,cnaccounts,dcuta,dcedu" \ -w
"ltpasswordgt" \ -b "cnpeople, dcuta, dcedu" \
"utaAccountNamedoran"
ltuidgt
LDAP server hostname
Bind Distinguished Name
Bind Credentials
Search Base
Search Filter
26
ldapsearch output
cedarid1085691573581181000,cnPeople,dcuta,dced
u utaAccountNamedoran utaPrevAccountNamedoran ut
aPrimaryAccountdoran utaPrimaryEmaildoran_at_uta.ed
u utaHomeStreet1123 Elm St. utaEmployeeTitleLibr
arian utaDeptCode6800000 utaEmployeeStatusactive
utaEmployeeCampusBox19497 eduPersonPrimaryAffili
ationstaff utaHomeCityArlington utaEmployeeTitle
Code0100 snDoran cedarid1085691573581181000 uta
HomeStateTX homePhone1 817 555
1234 utaEmployeeJobCategoryprof_non_faculty cnMi
chael D Doran utaEmployeeTypemonthly utaHomeZip7
6013 utaEmployeeBldgLIB givenNameMichael
search filter match
displayNamedoran, michael d utaEmployeeExemption
exempt utaEmployeePhone1 817 272
5326 homePostalAddress123 Elm St.ArlingtonTX76
013 utaID1000068572 utaEmployeePercentTime100.00
utaEmployeeRetirementCode9 utaEmplID1000068572
utaMiddleNameD objectClasstop objectClassperson
objectClassinetOrgPerson objectClassutaPerson o
bjectClassutaEmployee eduPersonScopedAffiliation
employee_at_uta.edu eduPersonScopedAffiliationstaff_at_
uta.edu utaPersonAffiliationemployee utaPersonAff
iliationstaff utaPersonAffiliationadministrative
eduPersonAffiliationemployee eduPersonAffiliatio
nstaff
patron-worthy search filter
27
LDAP search with Perl NetLDAP
!/usr/local/bin/perl -w use NetLDAP ldap
NetLDAP-gtnew("ldap.uta.edu") mesg
ldap-gtbind("uiddoran,cnaccounts,dcuta,dcedu",
password gt "ltpasswordgt") mesg
ldap-gtsearch( base gt 'cnpeople,
dcuta, dcedu', filter gt
"(utaAccountNamedoran)" ) foreach
entry (mesg-gtall_entries)
entry-gtdump ldap-gtunbind() exit(
0)
28
LDAP search with Perl NetLDAP
!/usr/local/bin/perl -w use NetLDAP ldap
NetLDAP-gtnew("ldap.uta.edu") mesg
ldap-gtbind("uiddoran,cnaccounts,dcuta,dcedu",
password gt "ltpasswordgt") mesg
ldap-gtsearch( base gt "cnpeople,
dcuta, dcedu", filter gt
"(utaAccountNamedoran)" ) foreach
entry (mesg-gtall_entries)
entry-gtdump ldap-gtunbind() exit(
0)
this Perl module will need to be installed
LDAP server hostname
Bind Distinguished Name
Bind Credentials
Search Base
Search Filter
29
Perl NetLDAP output
dncedarid1085691573581181000,cnPeople,dcuta,dc
edu utaAccountName doran
utaPrimaryAccount doran
utaPrimaryEmail doran_at_uta.edu
utaHomeStreet1 123 Elm St.
utaEmployeeTitle Librarian
utaDeptCode 6800000 utaEmployeeStatus
active utaEmployeeCampusBox
19497 eduPersonPrimaryAffiliation staff
utaHomeCity Arlington
utaEmployeeTitleCode 0100
sn Doran cedarid
1085691573581181000 utaHomeState
TX homePhone 1 817 555 1234
utaEmployeeJobCategory prof_non_faculty
cn Michael D Doran
utaEmployeeType monthly
utaHomeZip 76013 utaEmployeeBldg
LIB givenName Michael
displayName doran, michael d
... ...
30
Superuser valid patron filtering
!/usr/local/bin/perl -w use NetLDAP ldap
NetLDAP-gtnew("ldap.uta.edu") mesg
ldap-gtbind("cnlibrary,cnapp,dcuta,dcedu",
password gt "ltpasswordgt") mesg
ldap-gtsearch( base gt 'cnpeople,
dcuta, dcedu', filter gt
"((utaStudentStatusenrolled)
(utaEmployeeStatusactive)
(utaEmployeeStatusretired))" ) foreach
entry (mesg-gtall_entries)
entry-gtdump ...
a superuser Bind DN
patron-worthy LDAP records
31
Three easy(?) steps
  • Query the LDAP directory
  • filter for valid patrons
  • Process the records
  • minor data formatting
  • decision algorithms
  • output records in SIF format
  • Load SIF file into Voyager
  • Pptrnupdt

32
LDAP records simplified
dncedarid1085691573581181000,cnPeople,dcuta,dc
edu utaAccountName doran
utaPrimaryAccount doran
utaPrimaryEmail doran_at_uta.edu
utaHomeStreet1 123 Elm St.
utaEmployeeTitle Librarian
utaDeptCode 6800000 utaEmployeeStatus
active utaEmployeeCampusBox
19497 eduPersonPrimaryAffiliation staff
utaHomeCity Arlington
utaEmployeeTitleCode 0100
sn Doran cedarid
1085691573581181000 utaHomeState
TX homePhone 1 817 555 1234
utaEmployeeJobCategory prof_non_faculty
cn Michael D Doran
utaEmployeeType monthly
utaHomeZip 76013 utaEmployeeBldg
LIB givenName Michael
displayName doran, michael d
... ...
attributes
values
attribute-value pair
33
LDAP records made more complex
  • Not all records have all the same attributes

additional attributes for employees
additional attributes for students
generic attributes for all people
lets consider people records
34
Attributes continued
  • Some attributes are required, but many (most) are
    optional
  • Some attributes have a constrained set of
    possible values
  • Some attributes are multi-valued
  • Attributes will not necessarily
  • be sorted in any order

LDAP Schema
for information
35
Sample schema page (UTA LDAP)
36
Sample schema page (UTA LDAP)
37
Processing records
mesg ldap-gtsearch( base gt
'cnpeople, dcuta, dcedu', filter gt
"((utaStudentStatusenrolled)
(utaEmployeeStatusactive)
(utaEmployeeStatusretired))", callback
gt \ProcessRecord ) foreach entry
(mesg-gtall_entries) entry-gtdump

dumps records
we want to process records
The callback option allows you to process each
record as it is received
38
ProcessRecord subroutine
sub ProcessRecord my (mesg,entry) _at__
if (!entry) Finish(0) surname
entry-gtget_value('sn') first_name
entry-gtget_value('givenName') middle_name
entry-gtget_value('utaMiddleName')
uta_id entry-gtget_value('utaID')
_at_uta_affil entry-gtget_value('utaPersonAffil
iation') email_addr entry-gtget_value('
utaPrimaryEmail') home_phone
entry-gtget_value('homePhone') if
(home_phone) home_phone s/\1 //
home_phone s/ /\-/g ...yada,
yada, yada
39
ProcessRecord subroutine
sub ProcessRecord my (mesg,entry) _at__
if (!entry) Finish(0) surname
entry-gtget_value('sn') first_name
entry-gtget_value('givenName') middle_name
entry-gtget_value('utaMiddleName')
uta_id entry-gtget_value('utaID')
_at_uta_affil entry-gtget_value('utaPersonAffil
iation') email_addr entry-gtget_value('
utaPrimaryEmail') home_phone
entry-gtget_value('homePhone') if
(home_phone) home_phone s/\1 //
home_phone s/ /\-/g ...yada,
yada, yada
assigning LDAP attribute values to program
variables
single-valued attributes gt assigned to a scalar
multi-valued attributes gt assigned to an array
minor reformatting of data
40
LDAP record
Find the Voyager patron group
dncedarid1085691573581181000,cnPeople,dcuta,dc
edu utaAccountName doran
utaPrimaryAccount doran
utaPrimaryEmail doran_at_uta.edu
utaHomeStreet1 123 Elm St.
utaEmployeeTitle Librarian
utaDeptCode 6800000 utaEmployeeStatus
active utaEmployeeCampusBox
19497 eduPersonPrimaryAffiliation staff
utaHomeCity Arlington
utaEmployeeTitleCode 0100
sn Doran cedarid
1085691573581181000 utaHomeState
TX homePhone 1 817 555 1234
utaEmployeeJobCategory prof_non_faculty
cn Michael D Doran
utaEmployeeType monthly
utaHomeZip 76013 utaEmployeeBldg
LIB givenName Michael
displayName doran, michael d
... ...
The LDAP record doesnt have a patron group
41
Decision algorithms
patron group assignment example
program looks at student attributes first
if (stud_status eq "enrolled" stud_class)
if (stud_class /freshman/i
stud_class /sophomore/i
stud_class /junior/i
stud_class /senior/i )
patron_group "und" elsif (stud_class
/masters/i stud_class
/doctoral/i stud_class
/graduate/i) patron_group "grad"
expire_days "180" elsif
(stud_status eq "enrolled") patron_group
"und"
student classification not a required
attribute in LDAP record, so need a default
42
Decision algorithms
program looks at employee attributes second
if (empl_status eq "active") if (!
patron_group) patron_group
"staff" foreach my affil (_at_uta_affil)
if ((affil /faculty/i)
(affil /admin/i)
(affil /assistant/i))
patron_group "fac" expire_days
"400"
default only if no student patron group assigned
43
Decision algorithms continued
multiple sets of address attributes
work address phone
student address phone
home address phone
employees
students
all people
44
Three easy(?) steps
  • Query the LDAP directory
  • filter for valid patrons
  • Process the records
  • minor data formatting
  • decision algorithms
  • output records in SIF format
  • Load SIF file into Voyager
  • Pptrnupdt

45
A poetry break
Our time is fixed, and all our days are
number'd How long, how short, we know
not. Robert Blair, The Grave The SIF fields
are fixed, and all the fields are number'd How
long, how short, we better know. anonymous
programmer
46
Patron SIF formatting simplified
  • For each patron SIF record
  • Format each field
  • s string left-justified / blank-filled
  • n numeric right-justified / zero-filled
  • n special numeric if no value, then all
    blanks
  • d date YYYY.MM.DD format
  • Concatenate all fields into one long string
  • Slap an end-of-record marker on string
  • Write string to file

47
Remember this?
48
Formatting with sprintf
first_name sprintf("-20s",Michael)
patron_id sprintf("010d",123456)
concatenation first_name . patron_id print
concatenation Michael 0000123456
sprintf help
49
Three easy steps
  • Query the LDAP directory
  • filter for valid patrons
  • Process the records
  • minor data formatting
  • decision algorithms
  • output records in SIF format
  • Load SIF file into Voyager
  • Pptrnupdt

50
Things I didnt cover
  • Encryption
  • Data validation
  • Log files
  • Session file
  • Cron job setup
  • yada, yada, yada

51
You can do this!
Patron SIF
LDAP
protocol
you, or your resident geek
Perl programming
geek stigmata
52
Resources
  • Voyager Technical Users Guide
  • Patron SIF specification
  • Patron Update (Pptrnupdt)
  • Perl NetLDAP module
  • download from CPAN
  • documentation
  • Perl DateCalc module
  • download from CPAN
  • documentation
  • LDAP to patron SIF webpage (ldump.pl)

53
Questions?
Please fill out the session evaluation!
About PowerShow.com