Description
Question 1 [20 points]
Problem Set 4 1/31/19, 7(46 PM
https://jbhender.github.io/Stats506/F18/PS4.html Page 2 of 3
Use the Lahman baseball data previously seen in the SQL notes
(https://jbhender.github.io/Stats506/F18/Intro_to_SQL.html) to answer this question. Your answer should be a
single SQL query, but may require anonymous tables created using nested queries.
Write an SQL query to construct a table showing the all-time leader in hits (“H” from the “batting” table) for
each birth country (“birthCountry” in the “master” table). An all-time leader is the player (“playerID”) with the
most total hits across all rows (e.g. seasons/stints). Limit your table to players/countries with at least 200 hits
and order the table by descending number of hits. Create a nicely formatted table with the following columns
as your final output: Player (nameFirst nameLast), Debut (debut), Country of Birth (birthCountry), Hits (H).
Question 2 [40 points]
In this question you will modify your answer to Problem Set 3, Question 2 (PS3 Q2) to practice parallel,
asynchronous, and batch computing. Copy the functions from part a and c of PS3 Q2 to a new file
ps4_q2_funcs.R
In each of the parts below, let be defined so that
and be block diagonal with when and . (You may also use as in PS3 Q2 and
rescale in any other way that results in a positive definite .)
Create a table or plot for your results from each part.
a. Write an R script ps4_q2a.R that sources ps4_q2_funcs.R , and then uses mclapply to run parallel
simulations for .
Let and use 10,000 Monte Carlo replications. Reorganize the results into a long data frame
results_q4a with columns: “rho”, “sigma”, “metric”, “method”, “est”, and “se”. “Metric” should
contain the assessment measure: FWER, FDR, Sensitivity, or Specificity and “method” the multiple
comparison method used. The columns “est” and “se” should contain the Monte Carlo estimate and its
standard error, respectively.
b. Use your script from part a as the basis for a new script ps4_q2b.R . Setup a 4 core cluster using
doParallel and then use nested foreach loops to run simulations for and
. Reshape the results as before into results_q4b saved to a file
results_q4b.RData . Use a PBS file to run this script on the Flux cluster.
c. Modify your script from part a to create ps4_q2c.R which reads the following arguments from the
command line: sigma , mc_rep , and n_cores . Also modify the script to use the futures package
for parallelism. Use a PBS file to run this script as a job array for . Hint: see the answer
at this page (https://stackoverflow.com/questions/12722095/how-do-i-use-floating-point-division-inbash) for how to convert $PBS_ARRAYID to sigma .
β ∈ ℝ100
= { βi
.1,
0,
i ≤ 10,
else.
Σ Σij = ρβiβj i ≠ j Σii = 1 β
Σ
ρ ∈ {.25i}3
i= −3
σ = 1
ρ ∈ {.25i}3
i= −3
σ = {.25, .5, 1}
σ = {.25, .5, 1}
Problem Set 4 1/31/19, 7(46 PM
https://jbhender.github.io/Stats506/F18/PS4.html Page 3 of 3
Question 3 [25 points]
For this question you should use the 2016 Medicare Provider Utilization and Payment data available here here
(https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/MedicareProvider-Charge-Data/Physician-and-Other-Supplier.html).
a. Put the data into a folder ./data and then follow the instructions to read this data into SAS.
b. Use one or more data steps to reduce the data set to those rows with “MRI” in the ‘hcpcs_description’
field and where ‘hcpcs_code’ starts with a 7.
c. Use proc means or proc summary (as needed) to determine the MRI procedures with the highest
volume, highest total payment, and highest average payment among the procedures represented here.
d. Repeat part b-c using PROC SQL.
e. Export the results from “c” and “d” to csv and verify that they match. You do not need to produce a nice
table within your solution document.
