Moving your MOSS 2007 databases to a larger drive

I was doing a small Proof-of-Concept with SharePoint Server 2007, using one server for the database (running SQL Server 2005) and another server for SharePoint Server. The problem is that the SysAdmin that installed SQL Server not only installed it on the C: drive but also let SQL creating new database on the C: drive. As you can imagine, the C: drive ended up dangerouly low on disk space. We needed to move the data files to another drive (yes, there was another drive that was supposed to be used for data). Here are the steps we used.
WARNING: These instructions are provided "AS IS" with no warranties, and confer no rights. To be on the safe side, always perform a full backup before any major configuration change. You will need downtime on the server to perform these operations. Follow these instructions at your own risk.
1 - We confirmed there were no users on the server and properly shutdown the SharePoint Server. This makes sure there's no application hitting the database while we're moving the files around. If you SharePoint Server and SQL Server are on the same box, you will need to shutdown IIS and all SharePoint services before proceeding. Check a previous post on this blog to identify those services.
2 - We detached all databases. We also needed to disconnect current users (although SharePoint services were shut down, some connections remained there). You can check a previous post on this blog to identify those 8 or more databases involved. Make sure you take note of those, since you will need them later.
3 - We created a new folder on the E: drive, where we were moving the files to. Then we moved those 16 or more files (there was an MDF and an LDF file for each one of the 8 databases). We did not move any of the system databases, just the SharePoint-related ones.
4 - We re-attached the databases, point to the new file locations at E:\. SQL Server 2005 had no problem mounting the files from the new location.
5 - We reconfigured the default location for the SQL Server data and log files, so future database get created on the right location. We had to restart the SQL Server service for this configuration change to apply.
6 - We restarted SharePoint, which did not really care for the fact that the databases were now in a different drive. All SharePoint knows about is server and database names, and those did not change.
In the end, it was actually not that hard...

Jose Barreto - MSFT

I tried and successful, and you?

Configuring Single-Sign On for SharePoint

As you learned in a previous blog, Single Sign-On (SSO) enables SharePoint users to authenticate only once when they access applications through SharePoint sites. In this blog I will review the steps necessary to configure SSO in your SharePoint farm. Before we proceed with the configuration the following conditions must be met:

• Your server must belong to an Active Directory domain.
• Your server must be connected to a domain controller.
• You must use a domain user account (not a group account).
• You must use a SharePoint server farm account.
• You must be a member of the local Administrators group on the encryption-key server (which is the first server where we will start SSOSrv).
• You must be a member of the Security Administrators role and db_creator role on the computer running SQL Server.
• You must belong to the single sign-on administrators group.

Setting Up and Starting the Single Sign-On Service

1. Go to the Start menu and select All Programs –> Administrative Tools –> Computer Management.
2. Expand Services and Applications.
3. Click Services.
4. Locate Microsoft Single Sign-On Service and rt-click.
5. Select Properties.
6. On the General tab of the properties window, click the Startup Type drop down menu and select Automatic.
7. On the same tab, under Service status, click the Start button.
   
8. Click Ok to close the properties window.

Configuring the Single Sign-On ServiceÂ
You will need to perform these steps on each server in your server farm. Once the service is started we can now go configure SSO settings in Central Administration.

1. Open the Central Administration web application.
2. Navigate to the Operations page.
3. In the Security Configuration section, click Manage Settings for Single Sign-On.
   
4. On the Manage Server Settings page, click the Manage Server Settings link.
   
5. Enter the Single Sign-On administrator account name using the domain\username format. Note: The group or user specified here must meet all of the following criteria:
   • A Windows global group or individual user account – cannot be a domain local group account or a distribution list.
   • If a user is specified, the user must belong to the same account as the SSO service account and the configuration account for SSO.
   • If a group is specified, the SSO service account must be a member of that group and the configuration account for SSO must be a member of that group.
   • They must be a member of the Farm Administrators group on Central Administration.
6. Enter the Enterprise Application Definition Administrator Account. (This user or group must be member of the SharePoint Readers group on Central Administration).
7. In the Database Settings Section, enter the NetBIOS name of the single sign-on database server.
8. Enter the Database Name.
9. Enter the Time Out settings. The default value is 2 minutes.
10. Enter the Delete Audit Log Records Older than value. The default value is 10.
    
11. Click OK.

Creating an Encryption Key

1. Navigate to the Manage Settings for Single Sign-On page.
2. In the Server Settings section, click the Manage Encryption Key link.
   
3. Click Create Encryption Key.
4. Check the New Encryption Key checkbox.
5. Click OK.

Backing Up the Encryption Key

1. Navigate to the Manage Settings for Single Sign-On page.
2. In the Server Settings section, click the Manage Encryption Key link.
   
3. Navigate to the location where you want to backup the encryption key. This must be a removable storage device.
4. Click Back Up.

Restoring the Encryption Key

1. Navigate to the Manage Settings for Single Sign-On page.
2. In the Server Settings section, click the Manage Encryption Key link.
   
3. Navigate to the location where you placed the backup encryption key. This must be a removable storage device.
4. Click Restore. 

Note: You should always backup the encryption key when you backup the database. In Part 2 I will explain how to configure account information for an Enterprise Application Definition.

In the previous blog you learned how to setup and configure Single Sign-On (SSO) for SharePoint. In the SSO environment, the back-end or external data sources are referred to as “enterprise applications”. An enterprise application definition needs to be configured for each external data source that SharePoint will connect to. After you have started the SSO service, configured Central Administration, and created and encryption key, you must configure account information for the enterprise application definition.

1. Navigate to the Operations page of Central Administration.
2. In the Security Configuration section, click the Manage Settings for Single Sign-On link.
   
   
3. Click Manage Settings for Enterprise Application Definitions.  

Sharee’s Blog

SharePoint Farm configuring and deployment

Part 1 – Architectural and Logical Planning

Planning and installing SharePoint Farm across enterprise network is not a trivial task. SharePoint is rarely installed in an isolated environment, and usually it interferes with the organization strategy and existing infrastructure. Many factors may affect farm design, performance, scalability and redundancy – from hardware devices in organization network, to network topology. As a result, leveraging and finding compromises among those factors helps to build consistent, reliable and flexible environment.
There are several whitepapers on the Microsoft TechNet portal describing requirements for SharePoint Farm, but most of them are either written without taking into account infrastructure scope or filled with irrelevant information that navigate the reader away from the problem scope.

In this document you will find the configuration recommendations regarding different SharePoint areas. All information is represented in the set of recommendations about different actions you need to undertake or pay additional attention when you install and configure your SharePoint environment. We tried to structure all section to follow the natural flow of SharePoint installation from the scratch – from pre-installing analysis requirements to post deployment actions.

We plan several whitepapers in our “Best Practices” series, and we are interested which topis you would like to see in our next SharePoint publications. Please send us your comments and suggestions via this form.

Introduction

Organizations adopting SharePoint face a variety of tasks – from planning, strategy, infrastructure and architecture design, UI Design, migration, and to development. All these tasks imply flexible infrastructural baseline before actual work starts. However, in reality we face the outdated environment and misconfigured farms that are not ready to implement new requirements. In such cases, baseline architecture becomes foundation stone of all SharePoint projects.
Why would we care about infrastructure and not about something else, for example development? Fixing infrastructure errors is very expensive task and leads into significant changes across SharePoint farm. For example, Index Role assigned to the wrong server and incorrectly configured Search can lead to performance and redundancy issues that might require up to 3 days fix. Development errors are not so expensive and can be fixed relatively quickly, but sometime such errors, eventually become infrastructure errors that lead to changes in infrastructure design.

“Architectural Planning”

Plan your farm and network communications before starting actual installation. The first thing to start is designing SharePoint architecture across corporate network. This includes understanding network structure, examining network devices and choosing the right SharePoint topology to fit the existing infrastructure and new requirements.

Examine corporate network

Start from description of the existing network design, location of all applications and system servers. Microsoft Visio 2007 and “Network Diagram” template is a good instrument for this task.

Record the location and information of corporate system servers, like Domain Controllers, File Servers, Mail Servers, Application and others. Dont’ forget about network services – firewalls, proxies, and etc. For example, locations of ISA Servers across corporate network – IP address, list of open ports and the administrative user.

The best way to maintain “Network Diagram” document is to update the single diagram that covers topology of all domains and how they are connected. The following diagram demonstrates the Visio document descibing the servers and devices across organization.

This diagram will give a holistic view of the existing topology and ensure quick access to information across different domains.

Examine network devices

All network devices in the topology play a vital role of how SharePoint performs and interacts among different servers. Information about locations and settings of all routers, switches, and accelerators become very important in planning server locations. For example, location of different WAN and XML accelerators across network affects SharePoint server organization and configuration.

Presence of different network devices affects the connection bandwidth and latency between farm’s servers, and thereby, affects the choice of appropriate SharePoint Farm topology. Network Load Balancers (NLB), routers and switches will affect how fast network response, therefore the farm should be designed with the least impact of these devices.

Refer to the following links for the detailed information about WAN accelerators, NLB and other network devices across SharePoint farms:

1. http://technet.microsoft.com/en-us/library/cc263099.aspx
2. http://blogs.msdn.com/joelo/archive/2008/01/17/global-sharepoint-deployment-partner-solutions.aspx
3. http://blogs.msdn.com/joelo/archive/2007/01/05/nlb-network-load-balancing-and-sharepoint-troubleshooting-and-configuration-tips.aspx

Network administrator is a friend

The IT administrator is the person who should participate in farm configuration from the very beginning. This person will be responsible for the configuration of all network servers and devices across corporate network.

Most of the SharePoint Farm topologies cross the bounds of domains and from the very beginning specific protocols and ports must be open. The best way to maintain current situation is to have a separate document, shared with administrator, with the description of protocols and ports to open across network services.

Detailed information about system accounts and list of ports is available in the following articles:

• Plan for administrative and service accounts (Office SharePoint Server) http://technet.microsoft.com/en-us/library/cc263445.aspx
• Office SharePoint Server security account requirements http://go.microsoft.com/fwlink/?LinkID=92883&clcid=0×409

Measure network latency

Network response time is one of the important factors that can affect SharePoint farm design. Ideally, you need to measure the latency between SharePoint servers and users in order to reorganize servers according the smallest response time.

Network latency is the key point to determine which of the proposed scenarios to implement in the current SharePoint deployment. (Latency is the time required for a packet to travel from one point on a network to another).

Use the Ping tool (ping.exe) to measure latency for:

• users – from the client computer to the Web server on the server farm;
• data centres that host servers of the same farm – from a Web server in the remote data centre to the database server in the primary data centre

Do not forget to divide the round-trip result by two, because all measures are one way only, not round-trip.

Compare results to the data below, and adopt environment to have latency lower those values.

Number of users	Concurrent users (10%)	Central Solution	Distributed solution
100-5,000	10-500	Bandwidth:   3+ Mbps (dual T1)Latency:   < 100 ms	Bandwidth:   1.5 Mbps (T1)Latency:   <100 ms
10,000	1,000	Bandwidth:     3+ Mbps (dual T1)Latency:   <250 ms	Bandwidth:   1.5 Mbps (T1)Latency:   <500 ms
100,000	10,000	Bandwidth:   3+ Mbps (dual T1)Latency:   < 250 ms	Bandwidth:   1.5 Mbps (T1)Latency:   <500 ms

The critical bandwidth for any SharePoint farms is 1.5 Mbps (T1) with 500ms latency. Overstepping these values will increase the page-load times dramatically, in 4 times at least. Refer to the diagrams in the “Plan for bandwidth Requirements” document, for more details about the bandwidth and latency results under different conditions.

Available network bandwidth and latency influences geographic deployments significantly. Data transfers across WAN links that span multiple cities, states, provinces, countries, or continents requires really fast lines to provide adequate response time, so design such topologies thoroughly.

More details for bandwidth requirements available in the following article http://technet.microsoft.com/en-us/library/cc262952.aspx

Become familiar with SharePoint farm communications

Before discussing servers’ redundancy and farm topologies let us review farm servers and how they communicate with each other. The following picture from from “Planning an Extranet Environment for Office SharePoint Server” TechNet article illustrates the communication channels within a server farm and which servers handle client’s request.

When a user issues a query, the query is sent to a Web server. The Web server communicates with the query server to build a list of results, and then communicates with the computer running Microsoft SQL Server to extend the list of results with summarization text, URLs, and security trimming. In parallel, the Web Server gets page data from SQL Server and renders them on fly. This diagram will help in understanding which roles to use on farm servers.

Plan a baseline topology

Analyse the existing infrastructure and plan a SharePoint topology for redundancy. The term redundancy is often misinterpreted to be synonymous with availability.

Redundancy refers to the use of multiple servers in a load-balanced environment for any of several purposes, such as to improve farm performance, to scale out to accommodate additional users, and to improve availability.
Availability is a more specialized concept that refers to a multiple-server environment that is designed to accept connections and operate normally even when one or more of the servers in the farm are not operational. Therefore, availability implies redundancy.

There are several different topologies – from three to six servers in farm, which can be used as a baseline. Which one to choose depends on the level of redundancy and available hardware. Not all clients can afford topology with six or ten servers in farm due to budget limitation or data centre capabilities. Finding the compromise between numbers of servers, type of hosting and servers’ roles become critical task, because this choice will affect performance and extensibility of the SharePoint farm for several years ahead.

Three Servers Farm

The minimum availability for the farm with few servers can be achieved with “3-servers farm” topology. In the current topology Web and Application Servers locate together on the one box and the database is on another box. The remaining, third, server gives a choice of which server role make redundant – Web server role or the database server role.

The farm with the two Web Servers provides redundancy of the Web and Application roles, improving the overall performance. A drawback of this design is that your data is not redundant (left farm). In other case, farm with two Database Servers (cluster) provides data redundancy, increasing availability of critical data, but users might suffer from temporary loss of access, when Web Server unavailable (right farm).

The “3-servers farm” is one of the most questionable farms in terms of redundancy and performance. This limitation in the number of servers cannot provide redundancy of Query Server and high performance at the same time.

Redundancy can be achieved with Query Roles on both Web App servers. In this case, Database Server is the only place for Index Role, but this will hinder the overall performance. The Index Role is very CPU and HDD consuming role and that is why database servers are not very optimal place for this role. Alternative solution is to assign Index Role to the Web Server with the Query Role, but this will not work effectively, because in this case, index will not be propagated to another Query Server in farm.

If performance is one of the priorities then consider using Query Server and Index Server Roles on different Web Application Servers. This is flexible design in terms of extensibility, because with the new servers in farm changing roles of Index and Query servers is not required.

Interestingly, a TechNet article (http://is.gd/8QbS, page 26) explains, that a Query Server can’t be used with Web Applications server for 3-servers farm. The reality is that, Web App and Query Role together are super common, more common than not (one of the reasons is that Query Server doesn’t use Network-Load Balancer – it uses its own algorithm).  What they actually mean in the TechNet article is that having the  Index on database server is not at all a recommended solution.

Four Servers Farm

Additional, forth server will add redundancy either for Data Server or for Web Server. However, it does not help much with performance. Current topology suffers from the same “3-servers farm” drawbacks – no place for Index Server with Query Role redundancy.

Five+ Servers Farm

The most common and highly available server farm topology is “5+ servers farm”, the farm with the middle tier server.

This middle tier server solves all issues of three and four servers topology by providing the dedicated tier for Index and Application roles. Additional servers in farm will extend middle tier, by assigning new roles to those servers – Excel Calculation Services Role, and Microsoft Office Project Server 2007 Role.

The following table summarize farm topology:

Farm Servers	Performance	Redundancy
3 – 4	Index on WFE with Query on another boxApp Roles on WFE	Index on Database, with Query on WFEApp Roles on WFE
5	App Roles on Middle TierDedicated Index Server on Middle Tier	App Roles on WFEDedicated Index Server on Middle Tier
6	Dedicated Web Server for Crawling, outside NLBDedicated Index Server on Middle Tier	App Roles on Middle Tier in NLBDedicated Index Server on Middle Tier

To optimize the overall performance of five and more servers SharePoint Farm, configure a dedicated Web Server for crawling content, especially when crawling a server farm that contains more than 500 gigabytes (GB) of content or crawling content over the WAN. To ensure that user requests are not affected by content crawling, remove the dedicated Web server from the network load balancing rotation. This is especially important in global environments in which the off-peak hours of a regional farm (when crawl jobs are likely to be schedule) coincide with the peak hours of the central farm.

Plan extranet topology

Choose the topology based on requirements for external users. This topology will provide a basis of network extensibility for applications servers and communications between them.

The simplest topology is “Edge firewall topology”, which is represented by following diagram, from TechNet article.

This topology applicable for the small farms, when there is no need to separate internal services from corporate network and secure communications between server farms. All remote users are separated from farm by ISA server which plays a role of remote proxy.

For the big farms, when security of communications is a priority, the recommended topology is “Back-to-back perimeter topology”. This is very flexible and adaptable topology for network changes.

The main advantage of this topology is that it isolates the server farm in a separate perimeter network. Layers logically separate all servers and communications are under control – any security damages affect only specific layer, not the entire farm. External user access is isolated to the perimeter network and users can be isolated in different AD for remote and corporate access.

There are some other extranet topology variations, but mostly all of them are based on “Back-to-back perimeter topology” with some modification.

Detailed information about farm topologies can be found in the following documents:

1. Best practices for My Sites: http://technet.microsoft.com/en-us/library/cc262706.aspx
2. Best practices for team collaboration sites: http://technet.microsoft.com/en-us/library/cc850694.aspx
3. Planning an Extranet Environment for Office SharePoint Server: http://technet.microsoft.com/en-us/library/cc262400.aspx

“Logical Planning”

Plan site collections

Plan number of site collections and sub sites in advance – content, location, security. Start with the single site collections and several sub sites rather then creating several site collections, and try to avoid new site collection if there are no requirements for this. The reason of such structure is that each new site collection works as a new application, with isolated scope to features, templates and search. Maintaining such structure is much easier than several site collections.

Organize site collection across several content databases

Do not end up with one big content database, because data optimisation will cause troubles in this case. For the small and development environments, single content database might be a preferable choice. However, for the large farms create several content databases and organize site collections among them. Having several content databases with sites helps to address the following:

• Keep content database size <100 GB, otherwise it could hinder performance (MS recommendation)
• Data usage optimization.
• Simplify farm backup and restoration.
• Flexibility for Disaster Recovery (DR) strategies.

More details about site collections in several content databases available in the following blog post: http://msmvps.com/blogs/laflour/archive/2008/10/14/tips-to-create-a-site-collection-in-new-content-database.aspx

Script actions

Prefer to script installation and SharePoint Farm configuration actions: setting roles, creating web sites and site collections, etc. Configuring successful farm from the first attempt has a change to fail due to complexity of SharePoint. Running scripts to repeat all actions will save time when something went wrong and new server installation is required.

Part 2 – Installation & Configuration

Installation

The recommended Windows environment that offers the best performance for SharePoint is to have 64-bit servers. Such an environment provides significantly larger address space than 32-bit one; more room for SharePoint assemblies, CLR/Native APIs, Network Stack, IIS/ASP.NET and other components hosted in their respective tiers.
Windows Server 2008 and SQL Server

SharePoint install on the Windows Server 2003 and 2008. The recommendation is to use Windows 2008 because it has the outmost security. For the Windows 2008 you need to activate the following roles: Web Server role and the Microsoft .NET Framework 3.5.

Take into account that Windows 2008 Web Edition is not supported for farm roles, except WFE boxes, due to restrictions of non-Web editions of SQL Server on Windows 2008 Web Edition. The release of “SQL Server 2008 Web Edition” extends usage of Windows 2008 Web Edition for SharePoint. Refer to licence regulations and SQL Server 2008 Web Edition info for more details: http://tinyurl.com/b8nype

Do not install SharePoint on Domain Controller box in virtualized environment. DC role of Windows 2008 limits performance of hard drive by turning off caching to provide AD consistency. Any installations on virtualized DC might not work properly.

SharePoint installation supports  SQL 2005 and 2008 Servers (even SQL Server 2000 is supported, but there are not much advantages of its usage). SQL Exress edition is supported as well, but for basic MOSS installation, however basic install of WSS 3.0 will use Windows Internal Database (WID doesn’t have size limitation). SharePoint installs only on SQL Server, but you can use BDC use content from 3-rd party databases.

A few advantages of SQL 2008 over SQL 2005 are in performance, encryption, clustering, mirroring and etc. Moreover, it provides updated SharePoint Web Parts for Reporting Services and KPI. Detailed information about SQL 2008 and SharePoint can be found in the following post http://tinyurl.com/cdkcyw.

User SQL Aliases

When provisioning a new SharePoint farm, it is highly recommended to use an alias to connect to the Microsoft SQL Server, as this provides for greater flexibility to move the SharePoint databases to a new server. For example, using an alias during the installation will simplify the migration process of SQL database server from small environment to larger physical cluster during scaling out process.

Install Microsoft Office 2007 on farm premises (optional)

Office 2007 is not required on SharePoint server, but it might be  good to have it somewhere in you farm premises (not server box) for administrators, especially when you outsource your support or/and admins connect  remotely. In this case they might  need client apps installed somewhere to have access to

• Word and Excel for documents in Document Library
• PowerPoint for Slides in Document Library
• Access for “Edit in Datasheet” support in Document Library

Consider the same for Office SharePoint Designer(SPD), which is necessary for customisation purposes. (SPD is a  free product with is distributed as separate product that is not include into Office)

Install all Windows Updates

Make sure that all servers have the latest service packs and updates for Windows, SQL and Office prior installing SharePoint.

Choose the right edition of SharePoint

Be careful when use Enterprise edition of SharePoint, because Microsoft doesn’t provide the support if you decided to downgrade to Standard version, due to loss in features and functionalities. If you need Standard edition then consider a fresh installation.

Install SharePoint

Install SharePoint across all servers in farm. Start with WSS/MOSS slipstream package (with integrated latest Service Pack) rather then using basic WSS/MOSS installation and applying Service Pack later.

Follow the next order of installation:

• Application server where Central Administration site will be hosted
• All front-end Web servers
• The index server (if using a separate server for search queries and indexing)
• The query servers, if separate from the WFE servers
• Other application servers (optional)

Consider using scripts to automate SharePoint installation and configuration for the large farm deployment. SharePoint provides several configuration files and console commands that will do all deployment un-attendant. This will speed up installation and brings consistency of building and rebuilding servers in farm.

There are three commands to automate SharePoint installation:

• SharePoint Setup.exe + Config.xml – to script the setup questions
• PSconfig.exe – to script configuration wizard
• STSADM.exe – to script central admin UI for creating web apps and site collections

Alternative solution is to use already preconfigured Power Shell scripts. “SharePoint Deploy” tool on CodePlex provides the configured scripts for the standard installation, which can be adapted to any environments.

Refer to the following documentation for details about unattended installation http://go.microsoft.com/fwlink/?LinkID=135694&clcid=0×409, and info about installations scripting http://technet.microsoft.com/en-us/library/dd335964.aspx.

Take into account that SharePoint does not uninstall properly and additional steps are required to clean remaining files and remove database tables, which are kept for the security reasons (http://nehasinha.wordpress.com/2008/02/01/uninstalling-moss-2007-manually). It might be better to reinstall everything from the scratch if something went wrong, including Windows Server – because it will save a lot of time trying to fix potential issues that might be caused by files from the previous installation. In this scenario, using virtualization and snapshots feature saves a lot of time.

Detailed information about SharePoint installation guidelines are published on TechNet http://go.microsoft.com/fwlink/?LinkId=106632

Check Office Web Service availability

Sometimes, SSL protocol of Office SharePoint Web Service is broken.  To test it, go to IIS Management Console and open SharePoint Office Web Service in browser via https://. If it doesn’t work – don’t process further till fix it. This is very critical stuff, because SharePoint roles cannot be assigned on other services in farm. (I’ve seen such issues across several clients, when you can’t use other boxes in farm and only Application boxes are available for Index and Query roles, because SSL was broken).

No known fix at this moment, except reinstalling SharePoint.

Install SharePoint updates

Install all SharePoint updates (Infrastructure Updates and/or Cumulative Patches) after farm is deployed. Check the release notes of the latest cumulative patch if it includes all previous patches and updates. Sometimes you need to install previous updates manually. Cumulative patch releases each 2 months. Follow the official documentation “Deploy software updates for Office SharePoint Server 2007” for the processes of how to deploy infrastructure update (WSS Upgrade needs to be installed first and only then MOSS upgrade).

Be careful with SharePoint hot fixes

There are several hot fixes, for example “Coreserver.msp” package, which are released after Cumulative Patch.  However, be careful with these fixes, because they are temporary solution before the next official update, and they are not properly tested. Install hot fixes for specific problems only.

Configuration

Enable SharePoint Features

Navigate to the Central Administration site and enable Enterprise SharePoint features, if necessary. The default settings is Standard features.

Assign roles to servers

Navigate to the Central Administration site and assign SharePoint roles across all farms servers, according the infrastructure design and topology.

Configure administrative tasks

Configure administrative tasks across servers, like email settings, blocked type, logging and etc. Setup SharePoint Shared Services and configure all related services like Search, Query, Application Services, Profiles and etc.

Disable “Central Administration” role

Navigate to the Central Administration site and disable “Central Administration” role for all servers in farm, except application servers. This action will disable “Central Administration” site and IIS won’t use additional resources to host this application.

Configure Warm-Up scripts

Install site “warm-up” scripts. Those scrips will compile each page of SharePoint site collections when box restarts or after IIS poor restarts. This script improves the response time when users request pages first time.

Those warm-up scrips use STADM command to “warm-up” the administrative interfaces and hit each page in the portal to force their JIT. The collection “warm-up” scripts available there http://blogs.msdn.com/joelo/archive/2006/08/13/697044.aspx

Recycle IIS application pool at different time

Make sure that the application pools are set to recycle at different times on different Web servers, in case of multiple Web servers in the farm.

Recycle different IIS Web sites at different times to avoid peaks on the Web servers. When recycling more than one application pool on a specific Web server at the same time, temporarily remove that Web server from the load balancer to avoid bad user experience.

Following these steps helps to install SharePoint and configure basis settings with the minimum amount of time and avoid most common pitfalls, which usually happens when SharePoint installs in the wrong order

Part 3 – Development Environment

Development Environment

SharePoint development environment configuration depends on the processes, type of engagements and type of work. The most popular solution that addresses the development scenarios is using local SharePoint farm, separated from the production servers, with the single installations of the SharePoint server on the development boxes. This provides isolation for builds, tests, and debugging across different teams, projects and production environments. The local environment is mostly isolated by development box and is installed on the host server or on virtual server. The following procedure is an overview of the steps that are required to create a typical SharePoint development environment.
Development Box installation

• Use Windows Server, Visual Studio, and SQL Server. Windows Server 2008, VS 2008 and .NET 3.5, SQL 2008, with TFS 2008 is officially supported environment for SharePoint development. Advantage of Windows 2008 is that it is fast in virtualized environments.
• Install SharePoint on development boxes and prefer not to connect to existed farm instances used on other stages. Development environment should stay apart, to develop and tests in isolated environment.

Chose development tools

There are varieties of tools that can make development fast and easy – from commercial to open source and Microsoft products.

Microsoft recommends to use “Visual Studio extensions for Windows SharePoint Services” (VSeWSS), which simplify code-up solutions for SharePoint (e.g. Web Parts, List Definitions, Site Definitions, etc) via UI and allows reverse-engineering existed site to extract definitions for SharePoint entities.

The disadvantages of VSeWSS are:

1) not intuitive for beginners;
2) doesn’t provide usability to change all properties of the features, and other SharePoint items easily
3) Cannot use VSeWSS projects without VSeWSS extension.

Others 3-rd party tools, which simplify development, are:

1. Management Tools

• SharePoint Spy (http://www.echotechnology.com)
• SharePoint Manager (http://www.codeplex.com/spm)
• SharePoint Analyser (http://community.bamboosolutions.com)

These tools help to manage services of SharePoint farm and get detailed information about configuration settings.

For example, screenshots of SharePoint Analyzer (left) and SharePointSpy (right)

2. Visual Studio Tools

• STSDev to create project files (http://www.codeplex.com/stsdev)
• SPSource to get site sources (http://www.codeplex.com/spsource)

These tools create Visual Studio projects with the SharePoint 12-hive structure, and provide build settings to build/deploy/retract WSP packages directly from IDE. STSDev is the best tools to create a variety of projects – from simple features and Web Parts to Custom Workflow projects:

3. Development Tools

• TypeMock Isolator – Writing test for the SharePoint is a hard task for the developers, due to number of internal, sealed and private classes that are used across all SharePoint objects. ”TypeMock Isolator” is the only unit-testing and mocking framework for SharePoint, which provide ability to fake recursively, and simulate collection easily.
• SPDisposeCheck – tool to check assemblies that use the SharePoint API to build the better code. It provides assistance in correctly disposing of certain SharePoint objects to help following the best practice.

4. Other Tools

• U2U CAML Builder – editor for CAML queries (http://www.u2u.net)
• SharePoint Feature – set of different development and debugging plugins (http://www.codeplex.com/features)

Such tools give your additional flexibility in everyday development stuff, providing different editors, log viewers and analysers to make SharePoint development easier and faster.

Setup Deployment environment

Configure Continuous Integration (CI) system to build SharePoint solution, compile output to WSP packages, prepare deployment scripts and deploy WSP to Test Environment. Team Foundation Server (TFS) is one of the recommended tool for this task. There are several articles describing how to adopt SharePoint solution for CI, setup builds and configure TFS Deployer to deploy SharePoint packages across different environments:
http://www.azsharepointpros.com/ShareAndEnjoy/Lists/Posts/Post.aspx?ID=10

As to SharePoint Visual Studio Templates, CodePlex has a “SPTemplateLand” project that provides “12-hive” structure for SharePoint projects and deployments.
This solution was slightly modified by Microsoft SharePoint Consulting Services guys to have single deployment package for multiple projects and support packaging additional SharePoint Artefacts (site definitions, root files). This project is published there http://blogs.msdn.com/gderun/archive/2008/01/11/sharepoint-visual-studio-project-and-deployment-templates.aspx

Configure testing environment

The following diagram depicts the most common development environment, which is recommended by “SharePoint Guidance patterns & practices” team.

1. Stand-alone SharePoint environment for development, unit testing and debugging of SharePoint project. Runs continuous integration and builds verification tests before deploying the SharePoint solutions to the test environment.
2. Source Control/Build Server to build SharePoint packages (WSP) and to deploy solution to test environment.
3. The test environment performs user acceptance testing, manual functional testing, automated functional testing, system testing, security testing, and performance testing. After the solution meets production requirements, the SharePoint solutions are deployed to the staging environment.
4. The Staging server uses to test the “production-ready” solution in an environment that closely resembles the production environment. The purpose of this environment is to identify any potential deployment issues. Although the Staging environment is optional for smaller applications where deployment failures are not critical

The staging environment represents the target production environment as closely as possible from the perspective of topology (for example, the server farm and database structure) and components (for example, the inclusion of the Microsoft Active Directory service and load balancing, where applicable).

Part 4 – Backup and Recovery Strategy

Backup and Recovery Strategy

Use separate server for Disaster Recovery (DR)

Use additional server with installed SharePoint for DR. Do not connect this server to the existing SharePoint farm – it should be isolated environment with single SharePoint installation, or “3 servers farm”. Disaster Recovery servers must locate outside corporate networks, and ideally, outside corporation premises. Usually, it locates in different data centre. The reason for this to provide data availability in case of the network or data centre outage.

Configure mirroring

Configure SQL mirroring of content databases to provide data reliability. It should be done in the early stages, when content databases are very small, not for the hundreds gigs of content. The reason is that network bandwidth to DR is usually limited due to distant location of the servers, and setting mirroring for large databases became slow process.
Refer to the following document for more details about configuring SQL mirroring http://go.microsoft.com/fwlink/?LinkId=83725&clcid=0×409

Recovery data immediately

Highly available farms imply reliable strategy of data recovery. SQL Mirroring is only part of this strategy. Data restore in reasonable time is one of the factor of DR strategy

Consider several scenarios for backup/recovery:

1. OOTB backup/restore. SharePoint does not provide reliable mechanism of automatic backups. Backup/Restore feature available via Central Administration or via STSADM command. The only way to schedule this task is to use STSADM and windows scheduler to configure backups/restore.
Disadvantages:

• not reliable approach due to dependency of singe instance of scheduler;
• scheduler location on the single server, no multiple instances;
• no granularity of restored items;

2) SQL snapshots of content database (http://tinyurl.com/d2auns)
Disadvantage - lack of granularity in backup/restore

3) Data Protection Manager (DPM) – is one of the recommended solutions for data recovery strategy in large SharePoint farms.

a. DPM provides reliable and flexible functionality to backup, search and restore items.
b. Supports backup/restore of different servers in network – SQL Server, Exchange, file shares backups, and etc.
c. Adapts to the changes in Farm dynamically – add, delete or relocate servers.
d. Monitors and records all changes in SharePoint content DB by integrating into SharePoint VSS Writer.
e. Provides search functionality for any deleted items across content DB and restore data from content db.
f. Granularity in restoration of documents, sites, and applications.

One disadvantage of DPM is that it requires additional, clean, SharePoint instance to be used as temporary location of restored items to merge them with production content DB.

Some recommendation of DPM usage:

- Do not use existed SharePoint farm servers disconnected from the SharePoint farm – this approach has many bugs, and restoration fails in most cases.
- Iinstall SP1 for DPM – it provides significant performance increase for indexes and catalogue (http://technet.microsoft.com/en-us/library/dd347836.aspx)

4) Third party tools – Quest Recovery Manager, DocAve and Idera Backup. They provide the similar functionality of the DPM, but in some cases address SharePoint behaviour only, and might not provide enterprise level of backup strategy.

Quest Recovery Manager is the preferable tool for data restoration – lightweight application that does not require additional servers to restore deleted items.

 

SharePoint Farm configuring and deployment. Part 5 – Virtualization

Virtualization

Virtualization in SharePoint farm is one of the key design factors that simplify server availability by providing number of additional servers that might not be available over physical server models, or solution become very expensive. Microsoft officially supports SharePoint farm in virtualized environment since mid 2007. The following virtualizations technologies are supported: Hyper-V, Virtual PC, Virtual Server and 3rd party providers like VMware.
One of the key factors for virtualization is that performance of virtualized farm is competitive to the physical farm. Microsoft tests shows:
- 7.2% less throughput on virtual Web roles with 8GB of RAM than a physical Web role server with 32GB of RAM;
- 4.4% slower in the page response time on the Hyper-V Web front-end than the physical server;
Virtualize SharePoint Web Role

Web front-end servers are responsible for the content rendering and have comparatively lower memory requirements and disk activity than other roles, what makes them is an ideal candidate for virtualization.

Choose disk type for Query Role

The query role is responsible for a search performed by users is a good candidate for virtualization. The disk type choice for this role depends on the size of propagated index and the rate of index propagation.
The recommendation for the large indexes and the farm with the high rate of the updated information to use a physical disk volume that is dedicated for the individual query server, rather than a virtual disk file.

Consider using Index Role on physical server

The Index server role in a SharePoint farm is often the most memory-intensive role, what makes it less ideal candidate for virtualization. Virtualized Index server role might be appropriate for development environment, small farm or farm with small content usage.
Take into account, that index can vary from 10% to 30% of the total size of the documents being indexed. For the large indexes (above 200 GB) consider using physical disk volume that is dedicated to the individual query server, rather than virtual disk.
For large farms with big amount of crawled data use physical Index server role due to large memory requirements and high disk I/O activity.

Do not virtualize Database role

SharePoint database role is the least appropriate role for virtualization in production scenarios, mainly due to the highest amount of disk I/O activity and very high memory and processor requirements. However, it is very common to see the SQL Server virtualized in test farms, quality assurance (QA) farms, and smaller SharePoint environments.

Do I need to virtualize Application role?

The decision of virtualizations the application roles, such Excel Services and InfoPath Services, depends on the roles usage. Those roles can be easily virtualized, because they are similar to Web Roles and mostly CPU intensive. When necessary, those servers can be easily moved to dedicated physical servers.

Virtualized scenario sample

The following picture demonstrates the common virtualized scenario of SharePoint Farm.

Common deployment scenarios for the SQL role in a SharePoint farm may have multiple farms, both physical and virtual, use a single database server or database cluster, further increasing the amount of resources consumed by the role. For example, in the picture above, the sample SharePoint environment illustrated maintains a two-server SQL cluster that is used by several virtual farms and one production farm.

Use proper number of CPU

Do not use more virtual CPUs than physical CPUs on the virtual host computer – this will cause performance issues, because the hypervisor software has to swap out CPU contexts.
The best performance can be realized if the number of virtual processors allocated to running guests does not exceed the number of logical processors (physical processors multiplied by the number of cores) on the host. For example, a four processor quad-core server will be able to allocate up to 16 virtual processors across its running sessions without any significant performance impact. Note that this only applies to sessions that are physically running simultaneously.

Use proper amount of RAM

Plan to allocate the memory on virtual sessions according the next rule – divide the total amount of RAM in the server by the number of logical processors (physical processors multiplied by number of cores) in the host server. This will align allocated memory along with NUMA sessions. Otherwise it will provide performance issues.
In some testing, a virtual SharePoint Web server role with an allocation of 32GB of RAM actually performed worse than a virtual server with an allocation of 8GB of RAM.

Plan to use physical drives

In virtual scenarios front-end Web servers or Query servers disk performance is not as important as it would be physicals servers of the Index role or a SQL Server database. A fixed-size virtual disk typical provides better performance than a dynamically-sized disk.
If disk speed is a high priority, consider adding physical drives to the host computer. Add new virtual hard drive and map it to an unused physical drive on the host. This configuration, called a “pass-through disk”, is likely to give the best overall disk throughput.

Consider using hardware load balancing

Hardware load balancing provides the best performance, comparing with the software load balancing. It offloads CPU and I/O pressure from the WFE’s to hardware layer thereby improving availability of resources to SharePoint. Examples of Hardware: F5 BIG IP, Citrix Netscaler, Cisco Content Switch. Software load balancing examples are Windows Network load balancing, Round Robin load balancing with DNS. It is a trade-off between cost and performance. (Additional details are here)

Be careful with snapshot feature on virtual servers

Using snapshots for the backup might cause you troubles, because SharePoint timer service might be  unsynchronized during the snapshot process, and once the snapshot is finished, errors or inconsistencies can arise. So, consider backups over the snapshots for the production environments.

Measure virtualized environment performance

After you complete your virtualized environment installation and configuration measure how fast you environment operates and optimize it to the best performance. Measure the following parameters:
1) Processor performance – “\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time” performance monitor counter.
Results: <60% Utilization is fine, 60%-89% – caution, > 90% significant performance degradation
2) Memory performance – “\Memory\Available Mbytes” of the physical memory available to processes running on the computer, as a percentage of physical memory installed on the computer.
Results: 50% is fine, 10% and below is critical
3) Disk performance – “\Logical Disk(*)\Avg. Disk sec/Read” or “\Logical Disk(*)\Avg. Disk sec/Write” disk latency on the Hyper-V host operating system.
Results: up to 15ms is fine, 15ms-25ms is warning, >26ms critical

Mode details about virtualized performance counters can be found in this post

Refer to the following document for more details regarding virtualization: http://technet.microsoft.com/en-us/library/cc816955.aspx

---

In the next, final part we will describe the post deployment steps, which should be done to clear farm from temporary stuff and make your farm ready for production.

SharePoint Farm configuring and deployment Part 6 – Post Deployment

Post Deployment

Manage all SharePoint ports

There are about 20 different ports SharePoint use and they need to be open across firewalls. Refer to “Planning an Extranet Environment for Office SharePoint Server” document for more details. It can be easier to have firewall disabled when configuring SharePoint, and then close all non-SharePoint ports when configuration is done.

Monitor how SharePoint operates

After finishing deployment the actual work for administrators and infrastructure guys starts. They need to monitor the health of SharePoint servers, status of web application and statistic of web sites usage, the growth of content database.

Microsoft recommends using “System Center Operations Manager 2007″ with SharePoint Monitoring Toolkit http://technet.microsoft.com/en-us/library/bb975149.aspx to monitor SharePoint farms. The only drawback is that it’s enterprise level tool and might not be an ideal solution for small SharePoint farms.

On the other hand “SharePoint Diagnostic Tool” is suitable for small farms that simplify the process of analyzing logs for troubleshooting, and helps significantly reduce the time to diagnose issues.

The commercial sector of 3rd party management tools for SharePoint is represented by Quest Manager, DocAve and others. Quest Manager is one of the best tools that provide detailed information about how SharePoint operates, visual statistic across web application and sites (see screenshot):

There are several free tools to manage large and small SharePoint farms with the review published on SharePointReview site: http://tinyurl.com/9p6j7e. One of the good tools is SharePoint Analyzer from Bamboo solution which provides flexibility to monitor status and diagnose errors.

To diagnose ULS logs the “SharePoint ULS Viewer” provides the most powerful features to sort, filter and search across ULS logs in different categories.

Add Central Administration to Trust/Local Zone

Windows 2008 enhances the security of the internet sites and in result Central Administration site does not play nice with the default internet settings – some links of Central Administration sections are hidden. The same behaviour exists when Central Administration site is opened under non-administrative account.
To fix this, navigate to the Internet Explorer Options -> Security Tab and “Central Administration” site to Trust zone (or Intranet Sites).

Manage size of Content DB

SharePoint performs the best on the content databases below 100GB. This is not a true limitation but rather a recommendation. SQL Server databases have been scaling far beyond 100GB for years now. Practically speaking, the recommendation is based primarily on two significant factors:
1) Service Level Agreement (SLA) requirements for a given organization may dictate that backup operations for the SharePoint databases must be executable in a limited amount of time. The size of the content databases will have a direct impact on how long it takes to execute that backup.
2) The storage subsystem must be robust enough to handle the disk I/O requirements of the SharePoint solution that it serves.
So, monitor how much database grows and create new database content after such threshold. The best way to achieve this is setting alerts via DBA.

Set SQL TempDB Data Files

It is a product team-recommended practice that the number and size of data files allocated for the SQL TempDB Data Files should be equal to the number and size of core CPUs presented on the SQL Server machine. Each data file should be equal in size. Optimal TempDB data file sizes can be calculated using the following formula:

[MAX DB SIZE (KB)] X [.25] / [# CORES] = DATA FILE SIZE (KB)

So, for the server with two quad-core CPUs and the 500 GB content DB there will be 8 files with 15 GB size per temp data file.

If possible, separate each data file to separate logical unit consisting of unique physical disk spindles

Maintain SQL Database

SharePoint databases are usually big, very big, so maintaining databases in primary task to provide the good performance (http://technet.microsoft.com/en-us/library/cc262731.aspx) Consider the following tasks:

• Checking database integrity.
• Defragmenting indexes by either reorganizing them or rebuilding them.
• Setting the fill factor for a server.
• Shrinking databases to recover unused disk space.

Create SQL Maintenance plan to schedule those operation being run automatically.

Manage security and permissions

For each site collection, there is a limited number of security principals (users and groups) that can be applied – 2000 per Web site (limitation of ACL). Use Active Directory security groups and SharePoint groups to manage access, instead of adding users individually. The more fine-grained permissions are applied, the more difficult to track who has access to what. Moreover, fine-grained permissions can affect performance because additional security checks must be performed for each item to which they are applied.

Remove unused and “ghost” users

During farm installation, configuring or migrating, SharePoint farm is usually end up with number of test users or permissions that were used for test reasons, or “zombifies” users – which survived migration and not assigned to nobody. It’s important to remove all these users and check  the permissions to lower the chance of security breach.
There are several tools that address current task – Access Checker, SharePoint SUSHI and “Deliver Point” which provide the best functionality to find “dead” users easily and check permissions hierarchy across all site collections and sites.

Plan windows updates

In a server farm environment software updates are not installed automatically, even if the Automatic Updates feature is enabled on Web servers. Windows Update Web site or the Microsoft Update Web site can’t be used for software update installation (http://technet.microsoft.com/en-us/library/cc263467.aspx). In this case planning of updates across SharePoint farm boxes is required. Recommendation is to test all updates properly on testing/staging environment with existed applications before propagating updates to production environment.

Adopt farm to number of users

If the environment serves approximately 5000 or more users, consider deploying a minimum of three front-end Web servers, or the number of front-end Web servers that are required to provide appropriate capacity plus one more. That way, when one front-end Web server is experiencing occasional high load, load balancers can route the traffic to other servers. In the case when there is only one remaining front-end Web server, this can cause a cascading outage in which the extra traffic that affected the first front-end Web server is transferred to the second front-end Web server, which brings it down too. Having an extra front-end Web server helps the farm to consume such loads and provide more performance and stability in the farm during such situations. Additionally, consider dedicating front-end Web servers to be configured for crawling reasons only. There front-end Web servers should be physical servers, not virtual.

Optimize index server

SharePoint crawls data incrementally, and index is provisioned the same way – by small portions. Usually new crawled data become available in 3-30 second. If new added file is not discoverable in such timeframe, it means that Index Server suffers from performance issues. Use USL logs and performance counters to investigate what is wrong. In virtualized environment use physical HDD or move Index Role to physical box.

Summary

Installing and configuring SharePoint farms never been a trivial task and usually characterized with number of different issues, which could lead to the reinstalling the farms from the scratch several times. Having proper plan and recommendation for the major milestones in the configuration provides benefits to have work done from the first attempt, and end up with the scalable and reliable environment prepared for the further extensibility.
Post Scriptum
I’d like to thank Darren Neimke, Jeremy Thake, Alex Meleta for their notes and recommendation regarding several chapters, and especially to Eric Shupps for noticing several errors and constructive criticism.  Special thanks to Neville Mehta for the inspiration to create and finish this whitepaper.

sharepointmagazine.net