Sometimes it is necessary to force additional observations into your datasets, inserting new observations into a dataset with the Data Step usually involves either creating ta new dataset containing the new observations using a  DATALINES statement and then appending this dataset with the original, or alternatively  using multiple OUTPUT statements  within a Data Step.

 DATALINES method

DATA class_in;
  ATTRIB name LENGTH = $8 sex LENGTH = $1;
  INFILE DATALINES;
  INPUT name sex age height weight;
  DATALINES;
Andy M 30 190 100
Ben M 60 180 110
Paul M 26 170 120
;
RUN;
PROC APPEND BASE = class DATA = class_in;
RUN;

Multiple OUTPUT statement method

DATA class;
  SET class;
  OUTPUT;
  name = "Andy";
  sex  = "M";
  age = 30;
  height = 190;
  weight = 100;
  OUTPUT; 
  name = "Ben";
  sex  = "M";
  age = 60;
  height = 180;
  weight = 110;
  OUTPUT; 
  name = "Paul";
  sex  = "M";
  age = 26 ;
  height = 170 ;
  weight = 120;
  OUTPUT; 
RUN;

PROC SQL INSERT INTO method

The DATALINES method is not ideal as it involves creating an additional dataset and ensuring that the variable lengths in the new dataset  are consistent with the length in the original dataset, otherwise truncation can occur. Whereas the Multiple OUTPUT statement method can be time consuming and error prone because it required one assignment statement to be manually typed out for each variable in each observation. The PROC SQL INSERT INTO approach provides a concise alternative to these methods. This approach inserts lines using the VALUES statement in combination with a comma separated list, as shown below:

PROC SQL NOPRINT ; 
  INSERT INTO class
  VALUES ("Andy", "M", 30,190,100)
  VALUES ("Ben", "M", 60,180,110)
  VALUES ("PAUL", "M", 26,170,120); 
QUIT;

If using this approach, care should be taken to ensure that the order of the values in the comma separated list is consistent with the order of the associated variable in the dataset. To minimise the possibility of making a mistake in the order, or if you want to populate certain columns only, the columns which you wish to update, and the order of the variables in your comma separated list can be specified in brackets after the name of the dataset, as shown below:

PROC SQL; 
  INSERT INTO class (name,weight)
  VALUES ("Andy",100)
  VALUES ("Ben",110)
  VALUES ("PAUL",.); 
QUIT;

The post SAS tips & tricks #11 – PROC SQL; INSERT INTO appeared first on Statskom.

“ERROR: Variable X not found” 

What causes the message?

This message occurs when a PROC Step  attempts to use a variable which is not present in the input dataset. This usually happens if, the variable was dropped or not kept in a preceding step, or if the code to create it has been accidentally omitted, or code has been copied from one place to another without making the necessary alterations.

What happens when this occurs?

When SAS cannot find a variable within a PROC Step, SAS will:

• outputs a message to a log stating the name of the variable(s) it cannot find, e.g. “ERROR: Variable X not found” implying that the variable X  is not present in the input dataset.
• stop the execution of the current PROC Step and not produce any output.

You will notice that SAS’ behavior here is different to uninitialized variables in DATA Steps as in that instance it continues to create the output dataset.

How to resolve it?

To resolve this issue, ensure that the not found variable is available in the input dataset to the PROC Step, or udpate the PROC Step so that it does not require the not found variable in order to execute.

Example

The following code attempts to sort the SASHELP.CLASS dataset by the variables AGE, BSA and BMI variables.

PROC SORT DATA = sashelp.class OUT=class;
  BY age bsa bmi;
RUN;

This code results in the following messages being printed to the log.:

 
 43         PROC SORT DATA = sashelp.class OUT=class;
 44           BY age bsa bmi;
 ERROR: Variable BSA not found.
 ERROR: Variable BMI not found.
 45         RUN;

 NOTE: The SAS System stopped processing this step because of errors.
 WARNING: The data set WORK.CLASS may be incomplete.  When this step was stopped there were 0 observations and 0 variables.

The lines:

ERROR: Variable BSA not found.
ERROR: Variable BMI not found.

Tell us that the BSA and BMI variables respectively were not present in the dataset SASHELP.CLASS.

While the line:

 
WARNING: The data set WORK.CLASS may be incomplete.  When this step was stopped there were 0 observations and 0 variables.

Tells us that as a result of the error, the output dataset was not created.
To resolve this issue, either the BSA and BMI  variables should be present in the input dataset or else they should be removed from the PROC Step.

For example, here we use a DATA Step to first derive the BSA variable and then remove the BMI variable from the PROC SORT:

DATA class;
  SET sashelp.class;
  bsa = ((height *weight )/ 3131 )**1/2;
RUN;

PROC SORT DATA = class OUT=class2;
  BY age bsa ;
RUN;

The post SAS tips & tricks #10 – ERROR: Variable X not found. appeared first on Statskom.

What causes the message?

This message occurs when a DATA Step  attempts to use a variable which is either not present in the input dataset or has not been created in a preceding step. For example, if the variable was drop or not kept in in the input dataset, or if the code to create it has been accidentally omitted.

What happens when this occurs?

When SAS encounters an uninitialized variable within a DATA Step, SAS will:

• outputs a message to a log stating the name of the uninitialized variable.
• set the uninitialized variable to missing, i.e. create a null numeric variable.
• continue to create the output dataset.

How to resolve it?

To resolve this issue, ensure that the uninitialized variable is available in your dataset at the point at which it is first required.

Example

The following code reads the SASHELP.CLASS dataset and attempts to create a new variable BSA_GRP, based on the value of a variable BSA:

DATA class;
  SET sashelp.class;
  IF bsa > 1 THEN bsa_grp = 1;
  ELSE IF bsa> 1 THEN bsa_grp = 2;
RUN;

This code results in the following messages being printed to the log.:

 NOTE: Variable bsa is uninitialized.
 NOTE: There were 19 observations read from the data set SASHELP.CLASS.
 NOTE: The data set WORK.CLASS has 19 observations and 7 variables.

The first line tells us that the variable BSA was uninitialized (not available) at the point in the DATA Step where it was first required. The remaining lines tell us that despite the uninitialized variable, the dataset WORK.CLASS is still created.

We confirm this by opening the output dataset, which will be as follows:

We see that SAS has created the output dataset, we also notice that it contains an empty numeric variable BSA.

To resolve this issue, either the BSA variable should be present in the input dataset or else it should be created in the current DATA Step prior to the statement in which the BSA variable is first used, for example:

DATA class;
  SET sashelp.class;
  bsa = ((height *weight )/ 3131 )**1/2
  IF bsa > 1 THEN bsa_grp = 1;
  ELSE IF bsa> 1 THEN bsa_grp = 2;
RUN;

The post SAS tips & tricks #9 – NOTE: Variable X is uninitialized. appeared first on Statskom.

In a similar way to character to numeric implicit conversions, this  message occurs when you try to use a numeric variable in a function that requires a character argument, or to assign a numeric value to a variable which has already been defined as numeric.

It is possible to rely on SAS to perform these conversions imlpicitly, however, in general this is not considered good programming practice and it is usually preferable to perform these conversions explicitly. Implicit conversions also take more CPU time than explicit conversions and so may cause your program to execute more slowly.

Consider the following example, where we try to concatenate the character variable type with the numeric variable horsepower.

DATA cars;
  SET sashelp.cars;
  type_horse = STRIP(type)!!"-"!!horsepower;
RUN;

The output is created however the log displays the following message.

 43         DATA cars;
 44           SET sashelp.cars;
 45           type_horse = STRIP(type)!!"-"!!horsepower;
 46         RUN;
NOTE: Numeric values have been converted to character values at the places given by: (Line):(Column).
       45:34

This tells us that the code in line 45 of the log, at column 34 numeric variable has been used where a character variable was expected. In our case, the concatenation function (!!) expects a character variable, however horespower is numeric.

As the output dataset has been created, it may be tempting to leave the code as is, and have SAS perform the conversion for you. However, printing the dataset shows why this approach can be dangerous, as shown below.

Note how instead of being concatenated directly onto the hyphen, the horsepower, there are many leading spaces prior to the digits. These leading spaces are created by the implicit data conversion. When SAS performs this implicit numeric to character conversion it must select the format to use when performing the conversion and the width of the converted string. By default, SAS uses a format of BEST12, since our horsepower values each contains three digits, it means that SAS pads the values with nine leading spaces when performing the concatenation.

This issue can be avoided by first using the PUT function to convert the values from numeric to character. As shown in the example below.

DATA cars;
  SET sashelp.cars;
  type_horse = STRIP(type)!!"-"!!PUT(horsepower,3.);
RUN;

The second argument of the PUT functions allows you to specify the format that should be used during the conversion. As the horsepower values all have three digits, we specify a format of 3., which means that the value of horsepower is written as a character string of three digits in the concatenation. This also removes the converted to warning message from the log.

Finally a prinnt of the dataset confirms that the leading spaces have indeed been removed.

The post SAS tips & tricks #8 – NOTE: Numeric values have been converted to charater values appeared first on Statskom.

NOTE: Character values have been converted to numeric values.

This  message occurs when you try to perform a numeric calculation using a character variable, or to assign a character value to a variable which has already been defined as numeric.

It is possible to rely on SAS to perform these conversions imlpicitly, however, in general this is not considered good programming practice and it is usually preferable to perform these conversions explicitly. Implicit conversions also take more CPU time than explicit conversions and so may cause your program to execute more slowly.

Here we look at a few example where this implicit conversion is performed and look at ways in which the programs can be updated to avoid the NOTE.

Consider for example the following DATA step. A character variable WEIGHT_KG is created and assigned the value “75″, the subsequent statement attempts to use this character variable in a numeric calculation. In this instance SAS will perform an implicit conversion (i.e. SAS automatically converts the variable from character to numeric so that the calculation can take place). Here the character variable “75″ is converted to a numeric variable and the statement executes giving the expected result, but writing a NOTE to the log explaining that SAS has had to perform an implicit conversion in order for the calculation to take place.

DATA weight;
  weight_kg = "75";
  weigth_st = 0.157473 * weight_kg;
RUN;

NOTE: Character values have been converted to numeric values at the places given by:
      (Line):(Column).
      35:26

To avoid the note being written to the log, you can explicitly perform the character to numeric conversion.

Going back to our example, this would involve using an INPUT function to convert the variable WEIGHT_KG to a numeric before the calculation takes place as follows:

DATA weight;
  weight_kg = "75";
  weigth_st = 0.157473 * INPUT(weight_kg,BEST.);
RUN;

Care should be taken with this approach however, in case the character variable contains values unsuitable for a numeric conversion. Consider the following example:

DATA vitals;
  weight = "70.5";
  OUTPUT;
  weight = "U";
  OUTPUT;
RUN;

DATA vitals2;
  SET vitals;
  weight_rnd = ROUND(INPUT(weight,BEST.));
RUN;

The second observation of this dataset would cause _ERROR_ variable to be set to 1 as the INPUT function tries to convert the character value “U” to a number. In this instance the log will show something similar to the following:

NOTE: Invalid argument to function INPUT at line 36 column 22. weight=U weight_rnd=. _ERROR_=1 _N_=2

Indicating that at observation 2 the variable WEIGHT contained a value “U” which was unsiotable for the INPUT function.

If the value of “U” is unexpected and perhaps indicates dirty data, then the best approach is usually to request that the input dataset is queried. If however the value is feasible, perhaps indicating that the test was not perofmed in this instance, then conditional programming can be used to indicate that the character to numeric conversion should only be performed on suitable input values, for example:

In the below example we use the ?? format modifier to first test to see whether the character variable contains a value which is suitable for conversion to a numeric, and then only perform the conversion if this is the case, otherwise a custom warning is written to the log indicating that the input data is not clean and that this should be investigated.

DATA vitals2;
  SET vitals;
  IF  INPUT(weight,??BEST.) NE . THEN weight_rnd = ROUND(INPUT(weight,BEST.));
  ELSE IF weight NE "U" THEN PUT "WARN" "ING: the following non numeric values were found in the WEIGHT variable: " weight "at observation:" _n_;
RUN;

The post SAS tips & tricks #7 – NOTE: Character values have been converted to numeric values appeared first on Statskom.

In this post we look at the SAS log message: “NOTE: Missing values were generated as a result of performing an operation on missing values.”, its causes and how to resolve it. This message is typically caused by attempting to perform a calculation on a numeric variable which is missing, its technical name is the propogation of missing values, since a missing input value causes a missing output value.

Consider the following vital signs data where we have three test results for pulse and two for heart rate.

DATA vitals;
  ATTRIB pulse1 pulse2 pulse3 hr1 hr2 format = best.;
  INPUT pulse1 pulse2 pulse3 hr1 hr2;
INFILE cards;
DATALINES;
60 70 65 71 .
65 . 70 73 74
63 . . 72 73
65 77 73 74 75;
RUN;

Notice that these test results are sometimes missing, now if we were to run a DATA Step similar to the below

DATA vitals2;
  SET vitals;
  pulse_sum=pulse1+pulse2+pulse3;
  hr_sum = hr1 + hr2;
RUN;

We would result in messages similar to the following being written to the SAS log:

79 DATA vitals2;
80 SET vitals;
81 pulse_sum=pulse1+pulse2+pulse3;
82 hr_avg = (hr1 + hr2)/2;
83 RUN;

NOTE: Missing values were generated as a result of performing an operation on missing values. Each place is given by: (Number of times) at (Line):(Column).  2 at 81:19 1 at 82:16

These messages tell you that the statement in line 81 of the log (81 pulse_sum=pulse1+pulse2+pulse3;) resulted in a missing values twice (2 at 81:19 )whereas the statement from line 82 (82 hr_avg = (hr1 + hr2;)/2 ) resulted in a missing value once (1 at 82:16) and a PROC PRINT of the results would show the following:

Obs    pulse1    pulse2    pulse3    hr1   hr2    pulse_sum  hr_sum

1       60        70        65      71      .       195        .
2       65         .        70      73     74         .      73.5
3       63         .         .      72     73         .      72.5
4       65        77        73      74     75       215      74.5

Note that as described in the log, the variable PULSE_SUM is missing twice (observations 2 and 3) whilse the HR_SUM variable is missing once (observation 1).

How you decide to deal with this message depends on whether you expect the variables in your input data to contain missing. If you do then you can omit missing values from your calculations by using the sample statistic functions. In this case we would use the MEAN and SUM functions as follows:

DATA vitals2;
  SET vitals;
  pulse_sum=SUM(pulse1,pulse2,pulse3);
  hr_avg = MEAN(hr1,hr2);
RUN;

Printing out the dataset gives the following:

Obs    pulse1    pulse2    pulse3   hr1    hr2     pulse_sum  hr_avg

1       60        70        65      71      .       195      71.0
2       65         .        70      73     74       135      73.5
3       63         .         .      72     73        63      72.5
4       65        77        73      74     75       215      74.5

Note that here PULSE_SUM is populated in every observation and is calculated as the sum of all available PULSE variables. HR_AVG is also populated in every observation and is calculated as the sum of all available HR variables divided by the number of available HR variables.

If however your data should not contain any missing values, then you could alternatively use the NMISS function to test whether your variables contain missing values and output a message to the log if they do.

More information on the mean and sum functions can be found in the Functions and Call Routines section of the SAS doc

The post SAS tips & tricks #6 – NOTE: Missing values were generated as a result of performing an operation on missing values. appeared first on Statskom.

NOTE: The query as specified involves ordering by an item that doesn’t appear in its SELECT clause.

A quick google search of this message returns several thousand hits suggesting it’s a common problem.

Some clients may find it acceptable to leave this message in the log, whereas others may require that it is removed. This message is most commonly caused by the user attempting to sort a dataset using a variable which will not appear in the final dataset or attempting to create a macro list from a specific variable but to order it using the values of another variable. In both instances the program should function as expected however the unsightly note will be present in the log.

Below we will look at how to avoid this message in  each of these instances:

Here is an example of some code which attempts to sort a dataset using code not present in the final dataset, this code would create the ordered output dataset as expected however “The query as specified …” note would be present in the log:

PROC SQL;
  CREATE TABLE class  AS
  SELECT name 
  FROM sashelp.class
  ORDER BY sex;
QUIT;

An efficient and concise way to prevent the NOTE being written to the log would be to use Data Set Options on the output table as follows:

PROC SQL;
  CREATE TABLE class(DROP=sex)  AS
  SELECT name, sex 
  FROM sashelp.class
  ORDER BY sex;
QUIT;

Here is an example of the some code which attempts to create a space separated macro list NAMELIST holding the values of the NAME variable ordered by the SEX variable. Again this code would work as expected, however the NOTE would be present in the log.

PROC SQL;
  SELECT name INTO :namelist SEPARATED BY " "
  FROM sashelp.class
  ORDER BY sex;
QUIT;

An efficient way to avoide this would be to first sort the data and then to use a DATA Step to create the list instead, for example using code similar to the below:

PROC SORT DATA = sashelp.class OUT = class;
  BY sex;
RUN;

DATA _NULL_;
  LENGTH retain_var $1000;
  RETAIN retain_var ' ';
  SET class end=eof;
  IF _n_ = 1 THEN retain_var = STRIP(name);
  ELSE retain_var = STRIP(retain_var)||' '||STRIP(name);
  IF eof THEN CALL symput('namelist', retain_var);
RUN;

Please contact us if you have any SAS issues that you would like us to feature.

The post SAS tips & tricks #5 – NOTE: The query as specified involves ordering by an item that doesn’t appear in its SELECT clause appeared first on Statskom.

In this post we look at how we can use a simple piece of SQL to create a histogram which gives a quick overview of the number of variables in each domain.

The SASHELP.VCOLUMN dataset contains a list of all variables present in each dataset. Here we use the repeat function to output a “*” character for each variable in the dataset. The output of the query is printed to the active output destination.

PROC SQL;
   SELECT memname  , REPEAT('*',nvar) AS freq
   FROM sashelp.vtable
   WHERE libname = "SASHELP";
 QUIT;

The result is something similar to the below, which gives a quick insight into the number of variables in each dataset in the SASHELP library.

This SQL histogram technique has a wide varierty of other applications, for example here we use it to show the frequency of countries within each region in the SASHELP.DEMOGRAPHICS dataset.

PROC SQL;
  SELECT region , REPEAT('*',COUNT(*)) AS freq
  FROM sashelp.demographics
  GROUP BY region
  ORDER BY region;
QUIT;

The post SAS tips & tricks #4 – Visualising SAS datasets with an sql histogram appeared first on Statskom.

If you’ve ever wondered how to determine which variables are present in a dataset, which titles have been set, what format is associated with a particular variable or which ODS destinations have been setup, the dictionary tables could be what you are looking for.

The dictionary tables are a collection of automatically available, read only, tables accessible via PROC SQL. For example, the following SQL clause would create a dataset containing variable level metaedata for the SASHELP.FLAGS dataset, which would include, amongst other things, a list of all the variables present in the dataset, their lengths, formats, informats and more.

PROC SQL ;
  CREATE TABLE flag_var_meta AS
  SELECT * FROM
  dictionary.columns
  WHERE libname = "SASHELP" AND memname ="FLAGS" ;
QUIT;

Whereas the following code would create a dataset containing all TITLES and FOOTNOTES currently set within the session.

PROC SQL ;
 CREATETABLE title_meta AS
 SELECT * FROM
 dictionary.titles;
QUIT;

It’s easy to see how this information can be used to automate subsequent programming steps.

More information on the SAS dictionary tables is available in the SAS (R) online documentation:

The post SAS tips & tricks #3 – SAS dictionary tables appeared first on Statskom.

The additional functions are shown in the table below and each can be used to retrieve a specific piece of metadata:

The example below shows how to use VARTYPE and VARLEN functions to retrieve the type and length of variables.

DATA  _null_;
  dsid=OPEN("sashelp.class");
  CALL SYMPUT("name_length",VARLEN(dsid,VARNUM(dsid,1));
  CALL SYMPUTX("height_type",VARTYPE(dsid,VARNUM(dsid,"height")));
RUN;

Note that in the VARLEN statement we specify the column number of the variable we are interested in. In the VARTYPE example we retrieve the column number by first using the VARNUM function.

All of these functions appear under the SAS SAS File I/O Category. More information about these functions is available within the SAS (R) Language Reference: Dictionary

The post SAS tips & tricks #2 – Retrieving variable metadata within DATA Steps appeared first on Statskom.