Skip to content

Condo descriptives (mean and std) update - #153

Merged
TimCookCountyDS merged 5 commits into
masterfrom
tim_condo_descriptives_update
Apr 22, 2026
Merged

TimCookCountyDS merged 5 commits into
masterfrom
tim_condo_descriptives_update

Conversation

@TimCookCountyDS

Copy link
Copy Markdown
Contributor

Adds mean and std to the condo summary stats. (See the gen_agg_stats function in pipeline/03-evaluate.R)- Closes issue #147 .
Matches the condo descriptives to those of the Res-val model. (ccao-data/model-res-avm#458).

New outputs can be found in the model performance tables:
(Adding mean and sd to both lists produces 16 new columns in total):

From sum_fns_list:

┌─────────────────────────────────────────────┐
│ New column │
├─────────────────────────────────────────────┤
│ sale_fmv_mean / sale_fmv_sd │
├─────────────────────────────────────────────┤
│ prior_far_fmv_mean / prior_far_fmv_sd │
├─────────────────────────────────────────────┤
│ prior_near_fmv_mean / prior_near_fmv_sd │
├─────────────────────────────────────────────┤
│ estimate_fmv_mean / estimate_fmv_sd │
└─────────────────────────────────────────────┘

From sum_sqft_fns_list:

┌───────────────────────────────────────────────────────────────┐
│ New column │
├───────────────────────────────────────────────────────────────┤
│ sale_fmv_per_sqft_mean / sale_fmv_per_sqft_sd │
├───────────────────────────────────────────────────────────────┤
│ prior_far_fmv_per_sqft_mean / prior_far_fmv_per_sqft_sd │
├───────────────────────────────────────────────────────────────┤
│ prior_near_fmv_per_sqft_mean / prior_near_fmv_per_sqft_sd │
├───────────────────────────────────────────────────────────────┤
│ estimate_fmv_per_sqft_mean / estimate_fmv_per_sqft_sd │
└───────────────────────────────────────

@wrridgeway

wrridgeway commented Apr 22, 2026

Copy link
Copy Markdown
Member

@TimCookCountyDS 3 things will i'm looking at this

  • rather than link to the issue in the PR description, you can link it to this PR using the "development" drop down on the right side, that way the PR closes the issue when it's merged
  • can we remove your name from the PR title (Tim Condo is a great nom de plume)
  • can we get pre-commit passing

@jeancochrane

Copy link
Copy Markdown
Member

rather than link to the issue in the PR description, you can link it to this PR using the "development" drop down on the right side, that way the PR closes the issue when it's merged

Just adding that I like to do both of these things (noting the issue in the comment and linking the PR to the issue using the "development" dropdown)! Though I agree with Billy that linking the issue is most important in cases where there's a 1:1 relationship between a PR and an issue.

@wrridgeway wrridgeway left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this for condos!

Comment thread pipeline/03-evaluate.R Outdated
Comment on lines +24 to +27
# Increase the max size of globals exported to future workers, │
# since rlang │
# condition-handling functions can be large │

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this documenting something that's actually happening? If it is, could we clean this formatting up?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

O weird, I thought I removed that before I pushed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was just a note to self, will cut it out

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

@TimCookCountyDS

Copy link
Copy Markdown
Contributor Author

@TimCookCountyDS 3 things will i'm looking at this

  • rather than link to the issue in the PR description, you can link it to this PR using the "development" drop down on the right side, that way the PR closes the issue when it's merged
  • can we remove your name from the PR title (Tim Condo is a great nom de plume)
  • can we get pre-commit passing

Will do- for pre-commit: when I run pre-commit on the server (before I push), I use pre-commit run. Should I be using a different command?

@TimCookCountyDS TimCookCountyDS linked an issue Apr 22, 2026 that may be closed by this pull request
@TimCookCountyDS TimCookCountyDS self-assigned this Apr 22, 2026
@wrridgeway

Copy link
Copy Markdown
Member

For me after I've installed the pre-commit git hooks (pre-commit install) for the repo I don't even need to run pre-commit. git commit should trigger pre-commit, and you shouldn't be able to commit without passing.

Is that right @jeancochrane ?

@TimCookCountyDS

Copy link
Copy Markdown
Contributor Author

For me after I've installed the pre-commit git hooks (pre-commit install) for the repo I don't even need to run pre-commit. git commit should trigger pre-commit, and you shouldn't be able to commit without passing.

Is that right @jeancochrane ?

got - it- done :)

@TimCookCountyDS TimCookCountyDS changed the title Tim condo descriptives (mean and std) update Condo descriptives (mean and std) update Apr 22, 2026
@TimCookCountyDS

Copy link
Copy Markdown
Contributor Author

@TimCookCountyDS 3 things will i'm looking at this

  • rather than link to the issue in the PR description, you can link it to this PR using the "development" drop down on the right side, that way the PR closes the issue when it's merged
    done
  • can we remove your name from the PR title (Tim Condo is a great nom de plume)
    done
  • can we get pre-commit passing
    done (as far as I can tell)

--- squash and merge, @wrridgeway ?

@jeancochrane

Copy link
Copy Markdown
Member

For me after I've installed the pre-commit git hooks (pre-commit install) for the repo I don't even need to run pre-commit. git commit should trigger pre-commit, and you shouldn't be able to commit without passing.

Correct! We mention this in our pre-commit docs, though perhaps it could be called out more explicitly, since I've noticed it's a step that may people miss: https://github.com/ccao-data/wiki/blob/master/How-To/Use-pre-commit-to-lint-and-format-your-code.md#installing-pre-commit

@TimCookCountyDS

TimCookCountyDS commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

For me after I've installed the pre-commit git hooks (pre-commit install) for the repo I don't even need to run pre-commit. git commit should trigger pre-commit, and you shouldn't be able to commit without passing.

Correct! We mention this in our pre-commit docs, though perhaps it could be called out more explicitly, since I've noticed it's a step that may people miss: https://github.com/ccao-data/wiki/blob/master/How-To/Use-pre-commit-to-lint-and-format-your-code.md#installing-pre-commit

For me after I've installed the pre-commit git hooks (pre-commit install) for the repo I don't even need to run pre-commit. git commit should trigger pre-commit, and you shouldn't be able to commit without passing.

Correct! We mention this in our pre-commit docs, though perhaps it could be called out more explicitly, since I've noticed it's a step that may people miss: https://github.com/ccao-data/wiki/blob/master/How-To/Use-pre-commit-to-lint-and-format-your-code.md#installing-pre-commit

Thanks! I think my mental model was slightly off when I read those-

@TimCookCountyDS

Copy link
Copy Markdown
Contributor Author

@Damonamajor would you mind running this with build-and-run model and giving the outputs a quick once over? (I did + checked that the additional columns were added + populated for the new model run- see comment above for new columns and gave the model report a quick once over)- but would you mind a quick spot check just in case I missed anything?

(not that I expect any problematic changes- but good to have a 2nd set of eyes on it before I merge)

@Damonamajor

Damonamajor commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

https://github.com/ccao-data/model-condo-avm/actions/runs/24792538947

I presume we expect to see the changes in model.stats (or something like this) on aws and in the performance report?

@TimCookCountyDS

TimCookCountyDS commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

https://github.com/ccao-data/model-condo-avm/actions/runs/24792538947

I presume we expect to see the changes in model.stats (or something like this) on aws and in the performance report?

I didn't update the performance reports yet- so the main place I've been keeping track of the changes is updates to the model.performance table -
See comment for a list of the new columns #153 (comment)

If mean and std should also be propagating to stats (and aren't) lmk, and we can think through it - (Mean and std currently follow the exact logic of the other summary stats functions, so wherever those show up, mean and std should as well).

@wrridgeway

Copy link
Copy Markdown
Member

Seems fine to merge after Damon has confirmed the columns appear as expected.

@Damonamajor

Copy link
Copy Markdown
Contributor

Looks good to me. Just for reference these are the columns which are in the table for

SELECT * FROM "model"."performance"
where run_id = '2026-04-21-sharp-manasi'

but not in

2026-02-11-recursing-rob

sale_fmv_mean | sale_fmv_sd | sale_fmv_per_sqft_mean | sale_fmv_per_sqft_sd | prior_far_fmv_mean | prior_far_fmv_sd | prior_far_fmv_per_sqft_mean | prior_far_fmv_per_sqft_sd | prior_near_fmv_mean | prior_near_fmv_sd | prior_near_fmv_per_sqft_mean | prior_near_fmv_per_sqft_sd | estimate_fmv_mean | estimate_fmv_sd | estimate_fmv_per_sqft_mean | estimate_fmv_per_sqft_sd

@Damonamajor

Copy link
Copy Markdown
Contributor

https://github.com/ccao-data/model-condo-avm/actions/runs/24792538947
I presume we expect to see the changes in model.stats (or something like this) on aws and in the performance report?

I didn't update the performance reports yet- so the main place I've been keeping track of the changes is updates to the model.performance table - See comment for a list of the new columns #153 (comment)

If mean and std should also be propagating to stats (and aren't) lmk, and we can think through it - (Mean and std currently follow the exact logic of the other summary stats functions, so wherever those show up, mean and std should as well).

Model.stats doesn't exist. It's what I was guessing model.performance was called.

@Damonamajor Damonamajor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@TimCookCountyDS
TimCookCountyDS merged commit 7a02bbc into master Apr 22, 2026
9 checks passed
@TimCookCountyDS
TimCookCountyDS deleted the tim_condo_descriptives_update branch April 22, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Descriptives (mean, std) to condo model

4 participants