py-research

About this project

This repository is a collection of Python utilities to help you analyze & visualize data, automate workflows and manage knowledge while working in an interdisciplinary R&D project. More precisely, it serves these main puposes:

1. Extend existing, well-established packages (e.g. pandas, numpy sqlalchemy, structlog, …) with small helper functions and abstractions to provide lacking functionality or make them easier to use & automate.

2. Enforce best practices on scientific code to make it more reliable, quick to deploy, and easy to monitor.

3. Provide a framework for knowledge and information management backed by different data sources and databases, to which analysis / dataviz functions can tie in seamlessly.

Current status

This project is actively maintained, but still under construction. The modules in their current form are working and continuously tested, but documentation is still minimal and many components of the framework are not yet implemented. Release versioning follows the Semver Spec, so breaking changes in existing API will only happen with major version upgrades, of which there are expected to be a few before the framework is complete.

Furthermore, some of the current functionality may be outsourced into a separate repo and package with these upgrades.

How to install

py-research is available as a Python package on PyPI:

pip install py-research

Alternatively, you can install it directly from git via:

pip install git+https://github.com/cloudlane-one/py-research.git

Contents

• API References
  • src
    • py_research package
      • Subpackages
        • py_research.db package
          • Submodules
            • py_research.db.base module
              • Table
                • Table.db
                • Table.df
                • Table.source_map
                • Table.indexes
                • Table.from_excel()
                • Table.to_excel()
                • Table.filter()
                • Table.merge()
                • Table.flatten()
                • Table.extract()
                • Table.keys()
                • Table.get()
                • Table.columns
              • SingleTable
                • SingleTable.source_map
                • SingleTable.indexes
                • SingleTable.filter()
                • SingleTable.trim()
                • SingleTable.extend()
              • SourceTable
                • SourceTable.df
              • DBSchema
              • DB
                • DB.table_dfs
                • DB.relations
                • DB.join_tables
                • DB.schema
                • DB.updates
                • DB.backend
                • DB.load()
                • DB.save()
                • DB.describe()
                • DB.copy()
                • DB.extend()
                • DB.trim()
                • DB.filter()
                • DB.to_graph()
                • DB.keys()
                • DB.values()
                • DB.items()
                • DB.get()
            • py_research.db.conflicts module
              • DataConflictPolicy
              • DataConflicts
              • DataConflictError
            • py_research.db.importing module
              • TableMap
                • TableMap.table
                • TableMap.map
                • TableMap.ext_maps
                • TableMap.link_attr
                • TableMap.link_type
                • TableMap.join_table_name
                • TableMap.join_table_map
                • TableMap.id_type
                • TableMap.id_attr
                • TableMap.hash_id_with_path
                • TableMap.match_by_attr
                • TableMap.conflict_policy
              • RelDB
              • tree_to_db()
        • py_research.plots package
          • export_html()
          • plotly_to_html()
          • with_dropdown()
        • py_research.reflect package
          • Submodules
            • py_research.reflect.deps module
              • get_all_module_dependencies()
              • get_dist_requirements()
              • get_versions_on_pypi()
              • version_diff()
              • get_outdated_deps()
              • version_to_range()
              • semver_range_to_spec()
            • py_research.reflect.dist module
              • get_distributions()
              • get_module_file()
              • get_module_distribution()
              • get_file_repo()
              • get_module_repo()
              • get_project_urls()
              • get_py_inventory()
            • py_research.reflect.env module
              • env_info()
              • is_in_jupyter()
            • py_research.reflect.ref module
              • PyObjectRef
                • PyObjectRef.object_type
                • PyObjectRef.package
                • PyObjectRef.module
                • PyObjectRef.object
                • PyObjectRef.package_version
                • PyObjectRef.object_version
                • PyObjectRef.repo
                • PyObjectRef.repo_revision
                • PyObjectRef.docs_url
                • PyObjectRef.reference()
                • PyObjectRef.from_url()
                • PyObjectRef.to_url()
                • PyObjectRef.resolve()
              • stref()
            • py_research.reflect.runtime module
              • get_calling_module()
              • get_full_args_dict()
              • get_return_type()
              • get_all_subclasses()
      • Submodules
        • py_research.caching module
          • FileCache
            • FileCache.path
            • FileCache.max_cache_time
            • FileCache.function()
          • get_cache()
        • py_research.colors module
          • default_highlights()
          • default_scales()
          • ColorTheme
            • ColorTheme.highlights
            • ColorTheme.scales
            • ColorTheme.activate()
          • get_theme()
          • to_bg_color()
          • default_file_path
          • load_from_file()
        • py_research.data module
          • is_number_dtype()
          • to_boolean()
          • to_integer()
          • to_float()
          • parse_dtype()
          • gen_id()
        • py_research.enums module
          • StrEnum
            • StrEnum.parse()
        • py_research.files module
          • ensure_dir_exists()
        • py_research.geo module
          • GeoAlliance
            • GeoAlliance.EU
            • GeoAlliance.EU12
            • GeoAlliance.EU15
            • GeoAlliance.EU25
            • GeoAlliance.EU27
            • GeoAlliance.EU27_2007
            • GeoAlliance.EU28
            • GeoAlliance.EEA
            • GeoAlliance.G7
            • GeoAlliance.G20
            • GeoAlliance.APEC
            • GeoAlliance.BRIC
            • GeoAlliance.BASIC
            • GeoAlliance.CIS
            • GeoAlliance.OECD
          • GeoScheme
            • GeoScheme.country_name
            • GeoScheme.continent
            • GeoScheme.cc_iso3
            • GeoScheme.cc_iso2
            • GeoScheme.alliance
            • GeoScheme.world
            • GeoScheme.to_coco_scheme()
          • GeoRegion
            • GeoRegion.label
            • GeoRegion.scheme
            • GeoRegion.display_label
            • GeoRegion.exclude_already_covered
            • GeoRegion.get_label()
            • GeoRegion.to_country_list()
          • CountryScheme
          • Country
            • Country.label
            • Country.scheme
            • Country.to()
            • Country.data
          • countries_to_scheme()
          • expand_geo_col_to_cc()
          • merge_geo_regions()
          • match_to_geo_region()
          • gen_flag_url()
          • gen_flag_img_tag()
        • py_research.hashing module
          • gen_int_hash()
          • gen_str_hash()
        • py_research.intl module
          • DtUnit
            • DtUnit.year
            • DtUnit.quarter
            • DtUnit.month
            • DtUnit.week
            • DtUnit.week_of_month
            • DtUnit.day
            • DtUnit.day_of_year
            • DtUnit.hour
            • DtUnit.minute
            • DtUnit.second
            • DtUnit.millisecond
          • Format
            • Format.decimal_notation
            • Format.decimal_digits
            • Format.decimal_min_digits
            • Format.decimal_max_digits
            • Format.decimal_group_separator
            • Format.datetime_auto_format
            • Format.datetime_format
            • Format.timedelta_auto_format
            • Format.timedelta_resolution
            • Format.country_format
            • Format.fallback_to_translation
            • Format.na_representation
            • Format.merge()
            • Format.spec()
            • Format.auto_digits()
          • Args
            • Args.values
            • Args.all
            • Args.match()
          • Template
            • Template.replace
            • Template.wrap
            • Template.args
            • Template.apply_tmpl()
            • Template.apply_args()
          • Overrides
            • Overrides.templates
            • Overrides.vocabulary
            • Overrides.format
            • Overrides.merge()
          • Localization
            • Localization.loc
            • Localization.overrides
            • Localization.show_raw
            • Localization.locale
            • Localization.override_dict
            • Localization.get_overrides()
            • Localization.activate()
            • Localization.deactivate()
            • Localization.term()
            • Localization.text()
            • Localization.label()
            • Localization.value()
            • Localization.formatter()
            • Localization.format_spec()
          • get_localization()
          • iter_locales()
          • default_file_path
          • load_from_file()
        • py_research.ranking module
          • create_rank_series()
          • create_ranking_filter()
        • py_research.sql module
          • ColRef
          • Col
            • Col.cast_sqla_attr()
            • Col.column
            • Col.foreign_keys
            • Col.deferred
            • Col.deferred_group
            • Col.deferred_raiseload
            • Col.active_history
          • ColumnClause
          • Data
            • Data.columns
            • Data.name
          • Table
            • Table.sqla_table
            • Table.columns
            • Table.name
          • Query
            • Query.sel
            • Query.query_name
            • Query.schema
            • Query.name
            • Query.columns
          • DeferredQuery
            • DeferredQuery.func
            • DeferredQuery.schema
            • DeferredQuery.columns
            • DeferredQuery.subquery
            • DeferredQuery.name
          • Schema
            • Schema.schema_def
            • Schema.db_schema
            • Schema.default
            • Schema.name
            • Schema.tables
          • DBSchema
            • DBSchema.schema_dict()
            • DBSchema.default()
            • DBSchema.defaults()
            • DBSchema.metadata()
          • DB
            • DB.url
            • DB.schema
            • DB.tag
            • DB.validate
            • DB.activate()
            • DB.default_table()
            • DB.to_table()
            • DB.to_df()
          • query()
          • default_table()
        • py_research.stats module
          • dist_table()
        • py_research.tables module
          • TableStyle
            • TableStyle.dfs
            • TableStyle.cols
            • TableStyle.rows
            • TableStyle.name
            • TableStyle.format
            • TableStyle.css
            • TableStyle.header_css
            • TableStyle.hide
            • TableStyle.include
            • TableStyle.require
          • TableColors
            • TableColors.row_even
            • TableColors.row_odd
            • TableColors.header_even
            • TableColors.header_odd
          • ResultTable
            • ResultTable.df
            • ResultTable.styles
            • ResultTable.labels
            • ResultTable.widths
            • ResultTable.title
            • ResultTable.max_row_cutoff
            • ResultTable.font_size
            • ResultTable.default_style
            • ResultTable.column_flatten_format
            • ResultTable.show_index
            • ResultTable.table_colors
            • ResultTable.table_css
            • ResultTable.index_label
            • ResultTable.html_description()
            • ResultTable.to_styled_df()
            • ResultTable.to_html()
          • to_html()
          • html_to_pdf()
          • html_to_image()
        • py_research.telemetry module
          • TqdmHandler
            • TqdmHandler.tqdm
            • TqdmHandler.emit()
          • configure_logging()
        • py_research.time module
          • datetime_to_interval_series()

Indices and Tables

• Index

• Module Index

• Search Page