Questions tagged [data]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
5 views

need recommendatoin for handling MDM in Big Query

I have a large warehouse that I have multiple sources come into. the pipelines are completely done, and the system is working. however, quality in business fields used in Surrogate Key definitions ...
1
vote
0answers
10 views

Create (and make accessible) DBs to be accessed through API [closed]

We produce regularly some data sets that might be interesting for the proper audience. The DBs have some thousand enties and the entries have 15 fields in a tree form: { "item": { ...
0
votes
1answer
21 views

Is there any existing software which can read the data from LCD screen via camera and put it to CSV or any other machine readable format?

For example, I have cheap 1 euro Chinese thermometer like this and I want to make a graph of the temperature depends on the time of the day. This cheap device does not have any integration with PC/...
0
votes
0answers
5 views

Are there any good GUI tools for transforming JSON files to different shapes?

I'm looking for a visual tool that can transform JSON files in one shape, into JSON files in another shape. Right now I'm doing this by manually writing JavaScript functions that transform the data, ...
0
votes
0answers
21 views

I'm using the PathCheck contact tracing app--should I use something else?

I live in the continental United States. My goal is to use an Android app that will track and locally store my phone's location history, so that I can share it with health systems and other ...
0
votes
0answers
20 views

Tool to build test data for boolean logic

I am developing an app which allows users to search a database of candidates. The users are quite tech savvy, and so may input complex search terms, based on skills which the candidates may have. E.g: ...
3
votes
0answers
27 views

Best Key-Value database for intensive insertion

We are working on a project of type (intensive insertion) should able to handle millions of insertion per day. The Current situation we have two table one of them includes 2 billion record (Id & ...
0
votes
0answers
4 views

What tools may I use to get a daily automatized report of loaded databases?

I'm working with a start-up as a Data Engineer. We have python automatized scripts for loading data to db's through different VM's per project, so far we've been using Slack to get notifications when ...
0
votes
1answer
20 views

Open-source software with API to manage very dynamic data tables

I'd know if there's any open-source software with API aimed to handle a (virtually) infinite number of tables of semi-structured data. What I already know Relational databases We all know that the ...
0
votes
1answer
26 views

Open and visually browse large CSV files [closed]

Which software do you recommend to open and visually browse large (>2.5 GB) CSV files?
0
votes
0answers
11 views

Looking for Open Source OPC UA Web Server

I am looking to log tags from PLC into a database. I have software called 'ogamma logger' that will be used to connect to the OPC UA server. Ogamma logger will log data to postgreSQL database. I am ...
0
votes
0answers
6 views

What web service can store JSON data, what I can query from my webpage with JavaScript?

I would like to use some web data storage service, where I can register, and then I will be able to store small amount of textual data, namely JSON. It would be important, that the data can be updated ...
1
vote
0answers
11 views

Data entity definition language

I'm aware that most languages allow you to define data entities and the relationships between then in various ways; but I'm wondering if there's anything that's targeted specifically at holding this ...
1
vote
0answers
19 views

Software for categorizing data based on language and type

Currently, I'm using windows file explorer for categorizing. There are folders for years (2017, 2018, etc.) and each of them has sub-folders for types (type_1, type_2, etc.) and under each sub-folder, ...
1
vote
1answer
19 views

Best software for visualising data in a 3D space [closed]

I need to show a collection of data and would like to do this in a 3D space which could be interacted with by viewers. Can either be cloud based or local software.
0
votes
1answer
21 views

create a formatted data file filled with random values

I need to find a tool that i can use to create a large file of up to 5000 lines (or more). Each line must be formatted like such [30 Random Characters], [0-150] [Single Char (S,M,L)] [0-9][0-9]:[0-...
0
votes
0answers
7 views

Containerized Big Data Solution - Apache Hudi - Where to Start

Looking for a stable docker container that has Apache {Hadoop, Hive, Kafka, Spark, Zookeeper} installed and configured. Saw that Apache Hudi had a similar solution however it is in the incubation ...
0
votes
0answers
8 views

A GitLab/Kaggle for open data management?

We know, for code there is GitHub and GitLab. GitLab's major difference is that this platform is available as open source software you could run yourself. I've seen questions like "what is GitHub for ...
3
votes
2answers
110 views

Looking for an app to track all foods eaten and how I felt

I am looking for an app (It should work on Android and ideally on Windows or browsers) to be able to precisely track all foods eaten. I am not interested in anything too complex, and I do not want ...
0
votes
1answer
24 views

Is there a software, making questions with given data as answer to it?

Is there a software, making questions with given data as answer to it? I have a math project; I am writing a review of the "search of solution set of quintic equations". I have taken primary sources ...
1
vote
0answers
41 views

Recommendations For examining XML

Are there any tools out there that are used to view and examine through large XML files? I work for a small company as a Data Quality Analyst and a lot of my job is looking through large XML files ...
1
vote
0answers
24 views

Flash Card App that allows data tree linking between cards?

I'm currently studying Japanese Kanji: I've taken a large list of kanji (2300) and arranged them all into smaller groups with about 4 kanji on average per group, based on their similar looks etc. (...
2
votes
0answers
18 views

Gratis data file viewer with search for Windows (to view and search for text in any file)

When I encounter a file of unknown contents, I often open it up in a hex editor or text editor to look inside. The problem with a hex editor is that I'm usually much more interested in content that I ...
2
votes
3answers
87 views

What's an alternative to spreadsheets that has cells with defined type of value?

I don't like spreadsheets because they are not typed, any cell can have a value of any type. That can cause errors. I'd like to be able to define a type for any cell and the software would prevent the ...
0
votes
1answer
40 views

Is there any API which can provide information on different animal species?

I am working on a project (not a programming project) where we require a lot of information about different species such as average weight, lifespan, height, population numbers, endangered status etc. ...
0
votes
0answers
64 views

What is the best web scraping tool with a user interface / GUI?

I work for a small social entrepreneurship start-up that is compiling a large database of gender and international development data. We're looking for a web scraping tool (Mac compatible) with a user ...
1
vote
0answers
47 views

Splitting a KML file by year / location

I have a large KML file containing enough data to cause most programs including Google Earth to have problems. I would like to divide it by year. However, if it could detect when I left my home city,...
0
votes
0answers
12 views

Client Management System with user-friendly interface

Do you know of some self-hosted Client Management System to manage our clients data, so we can do things like searches, filters, reordering, grouping clients by criteria, etc? The most important ...
0
votes
0answers
9 views

How to create basic analytics for each user? Open-source options?

I have been looking into how to create custom analytics for each user of a music site. So the user portfolio will show some analytics such as: Sales Plays (total and for individual tracks) Profile ...
1
vote
0answers
12 views

Recommendations for Technology Stacks Needed to Build a Feature Store?

I am in the process of designing a simple but robust and scalable feature store architecture. So far, I've built a framework for creating custom feature pipelines in Scikit-Learn. Ideally, I want the ...
3
votes
0answers
35 views

Schema Mapping Tool

I have several flat file data sources (CSVs) that are extracted, transformed and loaded into relational tables. At some point in time, when I look at a table, I often have a hard time figuring out ...
0
votes
1answer
25 views

Software to clean unnecessary memory from my PC without making any kind of damage?

I do have a lot of unnecessary data in my C-drive, some of them are from previously installed software and some are just some temporary files... Can you suggest me some good software to clean them ...
0
votes
0answers
9 views

What recommendations do you have for AI to manage data in my company?

We have a bit of an issue with the quality of our data in my company where the data in a process passes through many systems. We have looked at and are using BluePrism for automating tedious manual ...
1
vote
0answers
7 views

Great scraping tools/ data mining tools for bulk of websites, extracting few emails from each?

I have been looking for good open source tools / softwares that can read certain patterns and scrape a bulk of provided websites... My use case is like this: I have a list of ~900 company websites ...
0
votes
0answers
20 views

I am looking for PDFs testing automation software

PDF Testing Compare data from flat files to PDFs Write logic to ensure PDF has correct data Ensure there must be correct logos, bar codes, fonts, margin etc. Please give me some recommendations
1
vote
1answer
23 views

Looking for alternative to email for sending automated weather alerts

We have a website that monitors weather conditions. A cron job runs frequently, which generates alerts if it's too windy/hot/whatever. We already have all this running with alerts being sent by email, ...
0
votes
0answers
9 views

Mapping Customers DB to my to my databse

I'm trying to connect to various customers and map their databases to my database. I need a tool that I can comfortably map their database column/keys to my keys. currently, I'm writing big mapping ...
0
votes
0answers
16 views

Software to show Tabular Data in Angular based front end application

I am looking for recommendation(both licensed and open source) for a 1. Tool/software to show large tabular data in my Angular application, 2. Showing the bar chart against each filter option. I ...
1
vote
0answers
4 views

Analyze Websites for Key Words

I'd like to analyze Github and other websites for the prevelancy of certain key words. How would I go about doing this? The solution must be able to complete the following: Check for the occurrence ...
0
votes
0answers
9 views

entity - relationship builder software

i'm looking for some kind of software that would let me create "entities" (people, places, things, etc) with different characteristics (to be set by me) and tags. Then the software should be able to ...
1
vote
1answer
36 views

Prepare dataset in CSV for machine learning?

I have a dataset consisting of CSV files (weather data). My issue is to clean the wrong values or empty values before using the dataset for machine learning. Right now I have to copy by hand the data ...
1
vote
0answers
7 views

NodeJS package to generate correlated data?

I am looking to create a synthetic database and for that I need to generate some ordinal data that has a correlation, say 0.5. I have seen packages in R to generate correlated ordinal data based on ...
0
votes
0answers
17 views

Recommendations for software, library or framework to delta large CSV Files

I am currently investigating what is out there that would quickly and efficiently compare and produce a delta from CSV Files. What I am dealing with is data that is outputted from Sybase ASE. These ...
1
vote
0answers
15 views

An Open Source or paid tool to generate mock data for unit testing, performance testing and mocking production data

I am looking for a data generation tool that can generate large amounts of data with following requirements. Ability to configure type of data that needs to be generated -- Data has lot of ...
0
votes
0answers
16 views

Azure Data Factory Labs (Collecting data from RestAPi)

I think that this question may have been asked previously, but this was a few years ago. Does anybody know if Microsoft have an online lab of sorts to be able to use Azure Data Factory. I am ...
1
vote
0answers
12 views

Software to get FOREX data in CSV format

Could you guys recommend me a software that can: Download today's FOREX data only. Forex Currencies must contain Peso, Yen, Chinese yuan (Please correct me if I'm wrong), Euro. (If it can be modified,...
1
vote
0answers
3 views

What software can one use to create a data graph out of a website?

i want to extract all the data from a website, and the data is rendered as a data graph. So how can i go about extracting it and saving to an excel or database?
1
vote
0answers
26 views

Open Source ETL to load ActiveMQ data into a column-oriented database

Can anybody suggest me an open source tool which can integrate with activemq as a source and load data into any columnar database management system like greenplum/cassandra etc. I have analysed ...
2
votes
1answer
37 views

internet data scraping tools

I'm a little distressed, for my job I need to get some info from a website, specifically http://www.museumsusa.org/museums/ , I need for every museum in America to get the name, address, phone number ...
2
votes
0answers
106 views

State synchronization in distributed systems

Are there any frameworks or technologies to synchronize State/data updated on one peer with all other peers in a reliable network? I'm looking for key-val data and State information that will be ...