SAS® (Statistical Analysis System) is a powerful tool widely used in statistical programming for clinical trial data analysis and reporting. Whether you’re working on clinical trials or handling complex data analyses, mastering SAS programming techniques and managing large datasets efficiently are crucial for successful outcomes. In this article we’ll explore:
By integrating these strategies, you’ll be well-equipped to optimise your SAS programming and tackle the challenges associated with big data.
Working in the life science industry provides numerous opportunities to learn valuable SAS techniques, often through training sessions. However, some of the most effective SAS tips come from collaborating with other programmers—whether seeking advice on coding challenges or reviewing programs written by colleagues. There are many simple SAS tips that can be applied to everyday programming tasks.
Whilst every programmer has their own preferred style of programming, what must be at the forefront of every programmer’s style is Good Programming Practice (GPP). Naturally programmers tend to follow their own internal style, adhere to Standard Operating Procedures (SOPs) and also code things in a certain way, often based on what is perceived as correct. However, a user should take some time to define their own style within the framework of compulsory/mandatory GPP, consistent at least for each individual program. It is important to avoid making significant sweeping changes in coding practices within a program, and potentially across a project.
Compulsory GPPs will include those from programmer’s company, the client and any other mandatory SOPs that must be followed. These will form the base layer of a programming style that can be built off. Beyond this base there are many other sources from which a programmer can draw ideas and inspiration from whilst still retaining or conforming to GPP.
A good example of third-party sources is PHUSE’s very own Good Programming Practice Guidance Document.
Other third-party sources include programming resources such as this one. Though other resources may cover some other topic the recommended practice within the resources can be considered GPP and influence how a programmer develops their programming style.
Base Step: Mandatory Practices from SOPs |
Third Party Step: Ideas and concepts raised by other sources. Including GPP in SOPs that are optional |
Personal Style Step: Looking at the preceding steps and deciding on one’s own final style whilst conforming to Good Programming Practice that the programmer wishes to follow tying ideas with the others already explored. |
Most programmers will have used the ‘IF THEN ELSE’ statements when working with conditional processing. Assigning one value for when the condition is true, and another value for when the condition is false. In these situations, lines of code can be reduced by instead using the functions IFC and IFN.
These functions can both create a new variable with assigned values based on whether a condition is true or false. IFC will create a character variable and IFN will create a numeric variable.
Below are standard templates for using IF THEN ELSE to create numeric/character variables, along with their counterpart in IFN and IFC:
*Standard IF THEN ELSE code (numeric); if condition then variable=value1; else variable=value2; |
*Equivalent IFN code; variable=ifn(condition,value1,value2); |
*Standard IF THEN ELSE code (character); if condition then variable='value1'; else variable='value2'; |
*Equivalent IFC code; variable=ifc(condition, 'value1' , 'value2') ; |
It is evident that IFN and IFC effectively cut down and rearrange the keywords from IF THEN ELSE code and change it into just one compact line of code.
For example both the following sets of code can be used to create the same variable:
if sex='Male' then sexcd=1; else sexcd=0;
sexcd=ifn(sex='Male',1,0); |
Either code can be used here and will produce the same result. In order to compare processing times equivalent code was ran five times for each method and recorded the average times. These were run within server SAS version 9.1.3 and showed very little difference in average processing times. However the IFN function has the advantage of requiring less code.
These functions are ideal when creating variables with 2 possible values.
There may be times when a specific character, such as superscripts or trademark symbols in a footnote, is required in outputs but is not directly accessible on the keyboard. These characters and many more are all accessible for use in SAS programs.
This table shows some of the symbols in SAS available to use, along with their corresponding byte number in SAS and a description.
Byte(i) |
Symbol |
Description |
153 |
™ |
Trademark sign |
169 |
© |
Copyright sign |
170 |
ª |
Superscript a |
174 |
® |
Registered trademark sign |
176 |
° |
Degree symbol |
177 |
± |
Plus or minus sign |
178 |
² |
Superscript 2 / squared |
179 |
³ |
Superscript 3 / cubed |
185 |
¹ |
Superscript 1 |
188 |
¼ |
One quarter |
189 |
½ |
Half |
190 |
¾ |
Three quarters |
247 |
÷ |
Division sign |
The full list of characters can be seen by running the following code. This will create a dataset with a variable containing each special character and a variable with each respective byte number in SAS.
data check;
do i=1 to 255;
sq=byte(i);
output;
end;
run;
Once the desired character is identified, %let can be used to create a macro variable for it. For example the following code will create a macro variable for the superscript a symbol:
%let supa=%sysfunc(byte(170));
To check this has worked properly, use %put to write the value of this macro variable to the log:
%put &supa;
The macro variable can then be inserted into the program. For example, the following is standard proc report code for creating a table, but using this macro variable to create a superscript value in the footnote. The highlighted code shows where the macro variable is used.
proc report data=final nowd headline headskip split=' | ' ps=23 formchar(2)='_';
column ('__' pdcdl col1 ord1 ord2);
define pdcdl / left ' ' width=60;
define col1 / center 'DrugA|6 mg SC';
define ord1 / order noprint;
define ord2 / order noprint;
title 'Table 14-1.3 Summary of Important Protocol Deviations';
compute after _page_;
line @13 72*"_";
line @2 ' ';
line @14 "&supa Other than Treatment Compliance/Test Article Administration";
line @14 'Note: Deviation categories are not mutually exclusive';
endcomp;
break after ord1 / skip;
run;
This will create the following output. The highlighted part shows what the footnote looks like from inserting the special character into it.
In this example, it is a simple process to find out the byte number of the desired special character.
Often in datasets issues, outliers and unexpected values may not be noticed straight away. There is a quick way to see all the unique values of a variable in a dataset by using the ‘where’ option from the data menu.
Although its primary use is to subset the data, it is also ideal for getting an overview of what values a variable can take.
The following steps show an example of how to use this option:
Open a dataset and select 'Where...' from the 'Data' menu
List of values is now visible
Using this option will then show the unique values of that variable, sorted in ascending order. This makes for an easy way to get an overview of the data.
For example, in demographics datasets, all the values of age can be examined. For a numeric variable like this, the minimum and maximum values can be observed, providing insight into possible outliers.
Another example is reviewing start dates in the adverse event dataset, as these can often be subject to issues. Examining this variable allows for the identification of incorrect or unusual values that may cause problems during programming.
This quick way of looking at data values can be used for saving time when checking for issues, outliers and unexpected values.
When working on PC SAS there are many helpful keyboard shortcuts available. For example, when commenting or un-commenting blocks of code, time can be saved by using the shortcut instead of typing it manually. When trying to find the matching pair in a set of brackets, using the shortcut eliminates the need to spend time searching manually.
These actions can be performed with ease by pressing a few buttons on the keyboard. The following table shows some of the shortcuts available to use:
Action |
Keyboard Shortcut |
Convert highlighted text to upper case |
Ctrl Shift U |
Convert highlighted text to lower case |
Ctrl Shift L |
Comment highlighted text |
Ctrl / |
Uncomment highlighted text |
Ctrl Shift / |
Collapse all sections of code |
Ctrl Alt – (on number pad) |
Expand all sections of code |
Ctrl Alt + (on number pad) |
Move cursor to matching bracket |
Ctrl ( |
Move cursor to matching DO or END statement |
Alt [ |
The full list of available shortcuts can be accessed through the menu:
Tools -> Options -> Enhanced Editor Keys
This will show a list of all commands that have a keyboard shortcut assigned to them, along with a brief description of what the command will do.
Ticking the box Show all commands will then also show the commands that have not been assigned any shortcuts yet. These can then be set as needed. For example, here are some of the unused commands which can be set a shortcut:
The Assign keys… button can be used to set a shortcut or to change an existing shortcut.
Switching text to upper/lower case and commenting/un-commenting code to be especially useful and a good time saver.
Working with large SAS programs usually presents complications and issues with the need for through programming which results in large amounts of code and large syntaxes. To handle these challenges and avoid common pitfalls, planning and sectioning can be applied to SAS programs.
Sectioning work is a common approach to any complex problem, so it is no surprise that it plays such a key role in handling large code. Programmers naturally make their problems into smaller, more manageable modules, to then tackle individually. As SAS is a procedural language, this modular approach is a natural part of programming, however modularisation [1] is a large topic in the wider world of computer science.
The first stage of planning large code should be to consider the types of sectioning that will be applied to the code by creating an outline of the pseudocode. Below are four of the major section types; procedural, variable, observation and input. A well laid out large code is likely to combine all four in some way, however certain datasets may contain one type of section more than the others. It will vary by dataset, and it is the responsibility of the author to find the right sections for their programming style and output dataset.
As mentioned above, the ideal code will mix together all these styles of sectioning. A common approach is to begin and end with procedural sections in the form of dedicated sorting and merging. A vital signs dataset is likely to use a large number of observational sections, whilst a subject-level dataset will be more suited to variable sectioning. Please note that while preparing a section plan is important, it will always be subject to change. Study-needs can evolve, and new data can pose new challenges. Be aware that structuring of the code also needs to be adaptable.
A good naming convention agreed upon early on can make a big difference in late stage debugging and updates. Below are six key rules for naming sections:
Using suffixes and prefixes for datasets and variables produced in certain sections, allows for easy tracking of workflows and sorting of the work library. Do note however that this rules out using numeric prefixes, as datasets cannot start with a numeric character in SAS.
Succinct sections and proper naming conventions for datasets makes it easier to manage the code as it gets bigger. Comparing and matching different names is easier to do when the name is shorter as many library viewers may only show the first 8 characters by default. An example of a good length and dataset name might be A_BBB_X, where A is a section name and BBB is additional information.
In the above rule an _X is placed at the end of the dataset name. This is to keep the datasets sorted in the work library and is a very widely used technique. However, note that the ordering of sections should be done with caution, if a new section must be added later in production, it could result in having to rename many datasets.
At first a simple ‘A’ ‘B’ and ‘C’ sectioning might seem ideal: it’s short, alphabetical and easy to remember. However, updates and debugging will be made difficult. Whilst sections are fresh in the minds of the production programmers in the first week of production, after a few rounds of quality checking some of the section names may start to seem meaningless.
Much like every aspect of code development, a choice of naming convention should be well documented in the comments in case the code needs to be updated, adapted for new purposes or rerun by another programmer. It will also make code review easier at later stage.
Later in this blog we'll discuss some potential solutions for automating naming conventions using macros that will help in keeping section and dataset labeling consistent throughout big code.
A section can begin with a comment, a macro call or even a bookmark. The section header is an excellent opportunity to document sectioning and naming decisions, and good documentation is at the cornerstone of good programming practice. There are many designs for section headers and dividers, formed from various comment formats. For large code, more information may be stored in a section divider than just a section header name, such as a description of the code within the section, the input and output datasets or any parameters to be used and updated. It can also be used to define section specific macros to help improve consistency throughout the section. Below are a few suggestions on ways that one could format section headers.
Figure 1: Suggestion for format of section headers
/*=====================*/ /* Main section */ /*=====================*/ /*description: */ /* */ /*=====================*/ |
*------------------; *Sub-section ; |
/*=====================*/ /* Main section */ /*=====================*/ %let section_name = LAB; %let input_ds = LAB1; /*=====================*/ |
/**Main Section Macro Call**/ |
/*==MAIN SECTION==*/ %let prefix = LAB; |
/*==================*/ /*===MAIN SECTION===*/ /*==================*/ %let section_name=LAB; %section_starting_macro; |
It should be noted that these examples use “forward slash comments”, i.e. /*…*/ for main section headers, and for “regular comments”, i.e. * …*; for sub-sections. This way, the programmer can easily comment out large chunks of code within sections and comment out entire sections if needed. Alternatively, to inactivate a section a large uncalled macro can be created around the section. Also, notice that the macro example uses macro comments (i.e. %*…;), which prevents the comment being written to the log.
Not only are section headers important but also the overall program header. The best programs will start with a program header containing all relevant information. Important things to include are information such as which external datasets are being read in, any outputs the program is creating, concise purpose of the program, among other things. Special consideration for big SAS code would be to include a small summary of how the program functions. A list of sections and purposes can be very helpful for future navigation. Should a programmer inherit or come back to a program after an extended period, a quick glance at the header should tell them exactly where to look in the program in order to adapt, debug or continue creating the program.
Different environments for SAS will have different functionalities designed to help organise the code. The list is extensive and will be constantly evolving as new versions of SAS are released, so this blog does not go through all features available.
However, it is important to investigate the features of the SAS environment that can benefit the code layout. For example, in SAS 9.4 for Windows, by default, the combination of shift and F2 keys creates a bookmark. As many bookmarks as needed can be set, and pressing F2 alone will jump through all bookmarks, allowing for scrolling between sections easily.
Ordering large code properly will not only improve the readability, but may also have a positive effect on the efficiency of the program. Much like working with big datasets, big code may result in a large amount of read and write processing that will slow down the run time, and in turn the programmer’s own productivity. Sorting and merging are the biggest culprits of this, and whilst they are a cornerstone of most SAS programs, they can be minimised by ordering and sectioning.
Starting big code with a procedural section focused on sorting is a very popular approach, known as data pre-processing. From a planning and sectioning perspective, it establishes naming conventions and indexes for the incoming raw data. A common mistake is not sorting uniquely at this stage, which leads to further sorting being required at a later stage. When starting with a sort procedure section, think whether further sorts will be needed later in the code, and if so, reconsider the initial sort to combat this.
proc sort data = rawdata.labs out = work.alb (where = (paramcd="ALB")) work.gluc (where = (paramcd="GLUC"))
work.sodium (where = (paramcd="SODIUM"))
nodupkey dupout = dups_labs;
by usubjid visit param;
run;
In the above example of a proc sort, it should be noted that there is a single input dataset, but 3 output datasets, one for each parameter. Due to advance sorting, all three separated datasets will be needed for the upcoming processing. By using the proc sort statement in this way, the program will only read in and sort rawdata.labs once, rather than three times. This is a common technique when handling big data in SAS, but it also works well for big code as the run time is improved, especially over multiple sorts of different raw dataset. In addition, there is a NODUPKEY and DUPOUT. Removing duplicates should be avoided, but by applying the following code will verify whether the duplicates dataset is completely empty:
proc sql noprint;
select * from dups_labs;
%if &sqlobs = 0 %then %put USER: rawdata.labs contains duplicates;
drop table dups_labs;
quit;
Or alternatively:
data _null_;
set dups_labs;
put 'USER: rawdata.labs contains duplicates';
stop;
run;
This makes it easier to spot issues in the code early on, as well as keeping the work library clear of unnecessary information. Many different suffixes can be used here, including but not limited to; USER, ISSUE, NOTE, WARNING and ERROR. However, the PHUSE GPP guidelines [5] suggest avoiding the WARNING and ERROR options, as they may cause confusion with SAS generated warning and errors. Additionally, a programmer may wish to add the date and initials to a log printed comment, if this is consistent with GPP’s. It is important to review the relevant programming SOPs within the organisation before finalising a method.
As a final consideration, if SQL is being used regularly or the programmer would like to implement more advanced techniques for handling bulk datasets, the SQL primary key system could be used. By defining or amending a table in a PROC SQL statement, it can state which variables should act as the unique sort order. This information is stored in the SQL dictionaries, and can be referenced by PROC SQL in the future. This has powerful applications for bulk sorting, merging and accessing [3][4].
An issue that programmers may come across when updating their code, is finding out that after updating one dataset name, unexpectedly later merges cease to work. As a code begins to branch out from the original trunk of the program, it may become harder to keep track of the various uses of every dataset. But how can this be planned for and fixed? What can be done to stop this issue in its tracks, or to fix the issue in hindsight?
Diagram 1
The first suggestion is perhaps a less sophisticated method, but also one of the easiest to implement. Take a pen and blog and begin to draw a simple flowchart or spider diagram, following the main chains of datasets and branching off as per the logic of the code. As SAS is procedural, the only deviation from a usual flowchart will be adding a macro, which can be included as a separate flowchart. Diagram 1 shows example illustration of this technique. Different layers will form naturally in the flow chart, which should line up with the sections of the code. Note that not every data step is listed, just enough for the flow chart to connect.
Alternatively, the log can be utilised to get a better understanding of the code. Using simple statements such as:
data _null_;
put “N””OTE AP: Section XXX has started”;
put “N””OTE AP: Input datasets = ”;
put “N””OTE AP: Output datasets = ”;
run;
Or:
%put %str(N)OTE AP: Derivation for variables XXX, YYY, ZZZ complete;
%put %str(N)OTE AP: Merged variable AAA for calculating BBB;
This code will output information to the log, with initials printed on the line. Then either manually or using a utility SAS program, a programmer can read the log with only the lines which contain their initials. Consider printing the section headers discussed earlier in this blog, with as much or as little information as is required for the complexity of the code.
The final suggestion is to produce one system of overarching macros which can track a dataset and the section names throughout the code. The section below discusses how this would work and shows a prototype of what such a macro could look like, along with the advantages and disadvantages this would offer.
The below macros are a proof of concept for ordering and numbering of dataset names. This macro may only be suitable in some cases and is not necessarily meant to be used in every program but is a further development of ideas already raised in this blog.
The idea of this macro is to generate a prefix, stored in the macro variable “&prefix”, for each dataset created in a program. When the work library is opened it will mean every dataset is stored in order of creation. This will allow ease of debugging and can allow programmers to create a narrative for their work library alone.
A condition for the following to work, each dataset name after the prefix macro variable within the dataset should be entirely unique. Every time a new dataset is created %prefix should be called. In addition, when a dataset needs to be read, then the second macro %prefix_wildcard should be called once. This macro searches the work library for the unique dataset name and stores the actual dataset name including prefix in a macro variable matching the unique dataset name.
Should the previous guidance be followed as specified in this blog regarding dataset naming, by being descriptive in the purpose and function of a dataset, then through the work library it will be clear what is going on without looking at the code. This turns a work library of a large SAS program that may have many datasets, from chaos to a refined ordered map of a program.
*==========================================================================; * Macro Define Section: Section in which both the %prefix and %prefix_wildcard ;
* macros are defined. ; *==========================================================================;
* The macro prefix creates a prefix for datasets created this will follow the form section_sequence-number eg the first prefix in section A will be A_001_unique_dataset_name;
%macro prefix;
%* Define global macros seqnumber prefix and section to create a unique prefix for a new dataset;
%global seqnumber prefix section;
%* If seqnumber macro variable already exists add 1 to it else create the macro variable as equal to 1;
%if %symexist(seqnumber)=1 %then %do;
%let seqnumber=%sysfunc(putn(%eval(&seqnumber+1),z3.));
%end;
%else %do;
%let seqnumber=1;
%end;
%* Create prefix macro variable;
%let prefix=§ion._&seqnumber._;
%mend prefix;
* The macro prefix_wildcard allows to essentially put a wildcard before a dataset name. This way every dataset should have a unique name then it will create a macro variable from the unique_dataset_name that has the prefix worked out;
%macro prefix_wildcard(unique_dataset_name=);
%global &unique_dataset_name;
%* Proc Sql to find the prefix of the unique_dataset_name in the sashelp.vmember dataset;
proc sql noprint;
select memname
into :&unique_dataset_name
from sashelp.vmember
where libname = "WORK" and memname like upcase ('%'||"&unique_ dataset_
name") ;
quit;
%mend prefix_wildcard;
*==========================================================================; * Section A: Section to show example of using %prefix and %prefix_wildcard macros.;
%let section = A; *==========================================================================; %prefix;
data &prefix.sdtm_input_1;
usubjid=1;
run;
%prefix_wildcard(unique_dataset_name = sdtm_input_1);
%prefix;
data &prefix.first_data_step;
set &sdtm_input_1;
run;
%prefix_wildcard(unique_dataset_name = first_data_step);
%prefix;
proc sort data=&first_data_step out=&prefix.first_data_step_sort;
by usubjid;
run;
When taking over and having to adapt a large SAS code it can be a daunting task. There could be any number or even all the following problems, ranging from the code being poorly commented, no comments at all, datasets being overwritten any number of times, unclear structure and poor indentation.
It is therefore important to do a quick read of the code to determine which course of action to take. A useful feature of SAS Enterprise Guide (EG) is the indentation tool, this can instantly make the code slightly more readable making skim reading a bit easier. If the working environment is not EG then code can be copied into EG then copied back into the working environment that is being used after indentation is applied.
Following a skim read, i.e. a quick read skipping some detail of the code, the decision to format and improve the existing code or start from scratch should be made. Below is the top level of each process:
Check for dataset overwrites, if they occur remove the overwriting. The first step of this could be just to add suffixes such as _1 and _2 to duplicated dataset names. |
If indentation hasn’t already been applied, while skim reading the code, it can be applied here. |
Separate the code out into sections and order data steps and procedures logically. |
Add comments, not only are these important for the finished code but they are helpful to personally keep track. |
Implement naming conventions of datasets. |
In addition to the challenge of working with large sas code, numerous SAS programmers experience problems when working with large SAS datasets that have millions of rows, hundreds of columns and are close to the size of a gigabyte or even more.
While managing large SAS code focuses on organising and structuring the logic for readability and efficiency, working with large datasets presents a different set of challenges. Large code is about handling complexity in the programming itself, whereas large datasets, with millions of rows and hundreds of columns, can strain system resources, slow down processing times, and require significant storage. These issues demand solutions focused on optimising data size, memory usage, and access speed. This section will explore techniques for working efficiently with large SAS datasets, focusing on minimising their size and improving performance.
Often it takes enormous amounts of time to process these datasets, running into issues such as insufficient space in file, which can have an impact on delivery timelines. Also, storing such datasets may cause shortages of permanent storage space or can exhaust the available memory on IS systems. Some tools which are often used to help lower the storage space required for datasets, i.e. compression/archive tools, may have a limit to the size of the data it can compress in the first place depending on the file system of the hard disk storing the data (e.g. FAT32 or NTFS), and the compression method used to compress/archive the data (e.g. ZIP or RAR).
These problems can occur when working with large datasets and integrating or pooling the data from various clinical studies. They can also occur with studies that have many laboratory parameters or with those with high numbers of patients.
In order to tackle these problems it is possible to make a large dataset smaller by reducing the number of observations and/or the variables, or by reducing the size of the variables without losing any of its information.
Another challenge is optimising data access. Indexing frequently used variables can reduce access time and improve performance. Moreover, large datasets may benefit from compression techniques, as SAS offers compression options like COMPRESS=YES to save space.
The solutions below are a few clinical programming techniques to reduce the size of your datasets and work with them efficiently.
A large SAS dataset can be made smaller by using SAS statements like LENGTH or dataset compression tools like COMPRESS option to save on the storage space. Reducing the size of the dataset can also help in reducing the time SAS takes to read or access data from a dataset.
Size reduction of a SAS datasets can also be achieved by the selection of required variables. For example, when using the DROP and KEEP functions for appropriate variables to include in the dataset and then remove the unnecessary variables which will reduce their size.
There is two types of compression in SAS CHAR and BINARY. CHAR compresses character variables more efficiently and BINARY works well for numeric variables and mixed-type datasets, offering greater compression.
The "YES | CHAR" compression type uses RLE (Run Length Encoding), which reduces storage by compressing repeated characters. For example, the string WWWBBHHHMMSS (12 bytes) becomes 2W2B3H2M2S (10 bytes), saving space. This compression algorithm is specifically designed for character data in SAS.
Here’s an example of how to use it in a dataset:
data work.temp (compress=yes);
set work.temp;
run;
The "BINARY" compression type combines RLE (Run Length Encoding) with sliding-window compression to reduce file size. For example, the phrase "count" repeated multiple times would be compressed using a small code to represent the repetition. While RLE alone cannot handle compressing repeated words, sliding-window compression efficiently encodes these repeated phrases. This method is especially effective for numeric data.
Here’s an example of applying binary compression in SAS:
data work.temp (compress=binary);
set work.temp;
run;
Another efficient method for working with large datasets is utilising SAS Views. SAS Views do not store actual data but rather the code to generate the data, refreshing every time the view is accessed. This method can be particularly useful when dealing with external files. Here's an example:
data work.temp / view=work.temp;
infile 'inputdata.txt' delimiter=';';
input pt dob weight;
run;
To view the structure of a SAS View:
proc sql;
describe view sasuser.faview;
quit;
This approach can help streamline data processing by reducing the need for permanent storage while accessing live data as needed.
Other techniques can be found through programming efficiency. This can be done in the form of saving storage space or memory, sub-setting observations and sorting the large datasets efficiently.
An example of coding efficiently can be seen here:
data work.temp;
length agecat 8;
set source.study1 (keep=pt dob);
if floor((dob-today())/365.25) lt 18 then agecat = 1;
if floor((dob-today())/365.25) ge 18 then agecat = 2;
run;
Is much less efficient than:
length age 8;
age = floor((dob-today())/365.25);
if age lt 18 then agecat = 1;
else if age ge 18 then agecat = 2;
run;
It is important to consider the displays that need to be generated ahead of time. By planning in advance datasets can be made into smaller sub datasets and therefore only need to use the smaller datasets. i.e. The Lab dataset is notoriously large, but saving them as 3 datasets “Lab – Blood Chemistry”, “Lab – Haematology” and “Lab – Other” means less manipulation required.
To save storage space or memory, some suggestions would be to:
It is recommended to perform data subsets early in the code. By sub-setting or filtering any unwanted records from a dataset, the size of the dataset can be reduced, thus saving storage space and improving efficiency of the program. For this purpose, either a WHERE statement or an IF statement can be used.
Perform sorting and sub-setting in the same SORT procedure to save storage space and processing time. Also, use the TAGSORT option in a PROC SORT command when there is insufficient disk space available to sort a large SAS® dataset.
To further enhance efficiency, SAS provides options like BUFNO and BUFSIZE to optimise how much data is transferred per read/write operation. The BUFNO option specifies the number of buffers to be used during processing, while BUFSIZE controls the size of each buffer. For example:
OPTIONS BUFNO=2 BUFSIZE=16384;
In this scenario, two buffers of 16,384 bytes each are used, meaning 32,768 bytes are transferred in one I/O operation. Note that BUFSIZE can only be used for output datasets and becomes a permanent attribute, while BUFNO applies only to the current step or session, offering flexibility for temporary adjustments. These options can reduce the time it takes to process large datasets significantly.
Mastering good programming practices in SAS is crucial for enhancing coding efficiency, especially when dealing with large datasets and big SAS programs. By applying the SAS tips and tricks covered in this article a programmer can significantly improve both the performance and maintainability of their programs. These SAS best practices help in managing challenges like insufficient space in file SAS environments and handling big data more effectively. By programming efficiently, the overall quality of the clinical trial reporting will improve which is beneficial for gaining regulatory approval of investigation new medicines in drug development.
References
[1] The Advantages of Modularization in Programming, G S Jackson - https://www.techwalla.com/articles/the-advantages-of-modularization-in-programming
[2] SDTM Model Concepts and Terms, Judy Li - http://pharma-sas.com/sdtm-model-concepts-and-terms/
[3] Create Index Guide, SAS Support - https://support.sas.com/documentation/cdl/en/proc/61895/HTML/default/viewer.htm#a002473673.htm
[4] Assessing SQL Dictionaries, SAS Support - https://support.sas.com/documentation/cdl/en/proc/61895/HTML/default/viewer.htm#a002473711.htm