Case Study: MS-DFS

Writing New Drivers to Extend

Likewise-CIFS

Gerald Carter
<[email protected]>
Director of Engineering
Likewise Software

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

Likewise Open – Background
http://www.likewiseopen.org/
 Likewise Open Project is the umbrella project sponsored by
Likewise Software designed to provide an interoperability
platform for non-Microsoft clients and servers in Microsoft
OS dominated networks.
 Likewise Open (product) refers to the open source
authentication & Active Directory integration suite
 Likewise-CIFS is the file server software stack
 Currently includes support for SMB and SMB2
 License
 Choice: Commercial or GPLv2+/LGPLv2.1+
 Single code base

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

2
Objectives

 Describe the Likewise I/O Manager driver model

 Explain the SMB/SMB2 SRV driver representation of
File Shares
 Show how SRV supports multiple File System
Drivers (FSD)
 Understand a standalone MS-DFS Namespace
 Learn how to configure shares to use a specific FSD

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

3
Likewise-CIFS Core Components

CreateFile Server Service Registry LwDsGetDcName Resolver

CreateNamedPipe (srvsvcd) (lwregd) (netlogond)
IoFsControl
DeviceIoControl
DNS
IPC (lwmsg) CLDAP

I/O Manager (lwiod) Likewise Security

Authority (lsassd)
Client File API
Client Lsa API
Core API Provider Routing
Driver API
Local AD DCE/RPC
Drivers Libraries LDAP
Accounts
Active Directory
Forest
2010 Storage Developer Conference. Likewise Software. All Rights Reserved.
4
Likewise I/O Manager

Likewise I/O Manager (lwiod)

Client File API
Core API
Driver API

POSIX Fs Object Caching

Clustered Fs
SMB/SMB2
Named Pipe Fs
Client
WebDAV Network
SMB Redir Requests
SAN Cloud
NFS
DFS

2010 Storage Developer Conference. Likewise Software. All Rights Reserved. 5

SRV (SMB/SMB2) Architecture

SRV SMB/SMB2
Listener Task tcp/445

Driver Socket Tasks

Worker Transport (select, epoll, etc.)

Protocol Response
Shares
SMB SMB2

Device IoControls
2010 Storage Developer Conference. Likewise Software. All Rights Reserved.
6
FSD and Shares

 Every driver registers a namespace

 E.g. “/pvfs” “/npfs” “/dfs” “/rdr” “/srv”
 The SRV driver embeds the FSD namespace within
the share’s absolute path
 Binds a share to a given FSD
 E.g. The share “Public” is bound to the I/O
Manager path on disk as “/pvfs/data/public”
 Client opens are relative to the Share root

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

7
Likewise DFS Driver

 libdfs.sys.so – Driver library loaded by lwiod

 Peered with POSIX and Named Pipe FSDs
 A DFS Root Share is a share bound to the DFS FSD
 All requests to the share are routed to the DFS driver
 Implements an ephemeral file system representing
 Standalone DFS Root
 A DFS Namespace held by a DFS Root
 Link Referrals for each Namespace entry

 GetDfsReferral SMB/SMB2 operations on IPC$

become DFS DeviceIoControls
2010 Storage Developer Conference. Likewise Software. All Rights Reserved.
8
MS-DFS Conversation
Client DFS Server Target Server
TreeConnect(\\server\dfs)
SUCCESS

CreateFile(\docs\expenses.xls)

PATH_NOT_COVERED

TreeConnect(\\server\IPC$)
SUCCESS
GetDfsReferral
(\server\dfs\docs\expenses.xls) SUCCESS
Referrals,
Path Consumed

TreeConnect(\\target\documents)
CreateFile(\expenses.xls)
SUCCESS

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

9
Likewise DFS – File System

DFS File System

 DFS Root Control Block
holds a list of referrals
DFS Root
 Referrals are displayed as
CtrlBlock directories from
IoQueryDirectoryInfo()
and contain a list of
Referral
CtrlBlock
referral targets
 Registry value name
Referral Target  Referral Targets are in
the form <UNC>:<TTL>
 Registry value data

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

10
DFS Registry Settings
[HKTM\Services\lwio\Parameters\Drivers\dfs\Roots\Standalone\Software]
"Documents“=sza:"\\server1\\docs:1800" "\\server2\\docs:1800"
"Public“=sza:"\\appserv.domain.com\\public\\software:1800"

Software

Documents

\server1\docs

\server2\docs

Public

\appserv.do…

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

11
DFS – Important IRPs

 IRP_TYPE_CREATE
 Must match a DFS_CREATE_CTRL_BLOCK or a
REFERRAL_CTRL_BLOCK and be a FILE_OPEN
 Partial namespace match returns PATH_NOT_COVERED
 Otherwise return OBJECT_PATH_NOT_FOUND

 IRP_TYPE_CLOSE
 Standard handle close

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

12
DFS – Important IRPs

 IRP_TYPE_QUERY_DIRECTORY
 If handle is to a ROOT_CTRL_BLOCK, then enumerate
referrals control blocks

 IRP_TYPE_QUERY_INFORMATION
 DFS Root Ctrl Blocks are reported as reparse points

 IRP_TYPE_DEVICE_IO_CONTROL
 Interface to GET/SET DFS_REFERRAL info structures
 SMBTrans2GetDfsReferral, SMB2_IO_CTL

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

13
DFS – Putting the Pieces Together

Likewise I/O Manager

Core API

Driver API CreateFile(/dfs/Software/iso/Likewise.iso)

IRP_CREATE tcon&X(\\host\mydfs)

PVFS SRV
ntcreate&X
(\iso\Likewise.iso)
NPFS “MyDfs” (/dfs/Software)

DFS “Public” (/pvfs/data/public)

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

14
DFS – Putting the Pieces Together

Likewise I/O Manager

Core API

Driver API CreateFile(/dfs)

DeviceIoControl(GET_DFS_REFERRAL)
IRP_DEVICE_IOCTRL
tcon&X(\\host\IPC$)

PVFS SRV
GetDfsReferral
(\host\mydfs\iso\Likewise.iso)
NPFS “MyDfs” (/dfs/Software)

DFS “Public” (/pvfs/data/public)

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

15
Caveat – Windows Share Mgmt

 Driver namespace is not exposed through the

NetrShareXXX() APIs
 MMC “Shared Folders” plug-in
 “Default” FSD is used for new shares
 Manual path configuration is possibly by directly
provisioning the registry
 [HKTM\Services\lwio\Parameters\Drivers\srv\shares]

 Administrative interface for standalone DFS Root

shares provided by an upcoming Likewise DFS-N
RPC service
2010 Storage Developer Conference. Likewise Software. All Rights Reserved.
16
Conclusions

 A multiple FSD driver model benefits developers by

 Allowing a feature or functionality to be isolated and
developed independently
 “I’d like to write a database FSD”
 Providing a “pivot point” against developers can verify a
new driver against a known working configuration without
recompiling or restarting
 “Things work fine with PVFS but fail with GOOFS.”
 Drivers can be loaded only when required and
therefore provide customized server builds
 Not limited to FSDs

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

17
Building Likewise-CIFS

 Simple build system for Linux & FreeBSD

 Step 1: Download the source code
 $ git clone git://git.likewiseopen.org/likewise-open
 Step 2: Build the likewise-open components
 $ build/mkcomp [--noincremental] [--debug] all
 Installs all pieces to “staging/install-root/”
 Step 3: Generate RPMs/DEBs (Linux only)
 $ build/mkpkg [--debug] all
 Creates packages in “staging/packages/”

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.

18
 Questions?

Gerald Carter [email protected]

Director of Engineering http://www.likewise.com/
Likewise Software git://git.likewiseopen.org/likewise-open

2010 Storage Developer Conference. Likewise Software. All Rights Reserved.